Golang Developer with Devops/LLM exp Job at Inherent Technologies, Remote

Y3lCVkRJdk8xcXhxTmxKejhzYzdzTGhBVlE9PQ==
  • Inherent Technologies
  • Remote

Job Description

Position: Golang Developer with Devops/LLM exp

Location: Remote

Duration: 1 Years

Job Description:

We are looking for devs with general cloud services / distributed services experience, with LLM experience as a secondary skill. GPU experience is now low on the list of preferred skills: Dedicated Inference Service

Required Skills-

  • Proficiency in Golang for building scalable and performant backend services.
  • Deep experience building services in modern cloud environments on distributed systems (i.e., containerization (Kubernetes, Docker), infrastructure as code, CI/CD pipelines, APIs, authentication and authorization, data storage, deployment, logging, monitoring, alerting, etc.)
  • Experience working with Large Language Models (LLMs), particularly hosting them to run inference
  • Strong verbal and written communication skills. Your job will involve communicating with local and remote colleagues about technical subjects and writing detailed documentation.
  • Experience with building or using benchmarking tools for evaluating LLM inference for various models, engine, and GPU combinations.
  • Familiarity with various LLM performance metrics such as prefill throughput, decode throughput, TPOT, and TTFT
  • Experience with one or more inference engines: e.g., vLLM, SGLang, and Modular Max
  • Familiarity with one or more distributed inference serving frameworks: e.g., llm-d, NVIDIA Dynamo, and Ray Serve etc.
  • Experience with AMD and NVIDIA GPUs, using software like CUDA, ROCm, AITER, NCCL, RCCL, etc.
  • Knowledge of distributed inference optimization techniques - tensor/data parallelism, KV cache optimizations, smart routing etc.

What You'll Be Working On-

  • Develop and maintain an inference platform for serving large language models optimized for the various GPU platforms they will be run on.
  • Work on complex AI and cloud engineering projects through the entire product development lifecycle (PDLC) - ideation, product definition, experimentation, prototyping, development, testing, release, and operations.
  • Build tooling and observability to monitor system health, and build auto tuning capabilities.
  • Build benchmarking frameworks to test model serving performance to guide system and infrastructure tuning efforts.
  • Build native cross platform inference support across NVIDIA and AMD GPUs for a variety of model architectures.
  • Contribute to open source inference engines to make them perform better on DigitalOcean cloud.

Job Tags

Local area, Remote work,

Similar Jobs

Minit Charger

Electronics Technician Job at Minit Charger

 ...demanding applications.Our solutions are at work at major airports across the globe and household names such as General Mills, Kellogg, Whirlpool, Good Year to name a few.We value great talent, dedicated and hardworking team members who play important role in mutual growth.... 

DSV - Global Transport and Logistics

Inventory Control Clerk Job at DSV - Global Transport and Logistics

 ...channels; to verify the implementation of solutions. Review/investigate issues with inbound and/or outbound functions including unloading, checking, orderly stacking, storing, order pulling, staging, labeling, and loading. Ship material to the factory upon client... 

Kelly Services

Greenhouse Technician Job at Kelly Services

**Additional Information:** Resumes can be directly sent to recruiter at ****@*****.*** **Greenhouse Technician, Johnston, IA 50131**Kelly Services is seeking a Greenhouse Technician for one of our top clients.**Duration:** 11/03/2025 to 10/30/2026**Type... 

The Boeing Company

SE&I National Team Systems Engineer (Lead and Senior) Job at The Boeing Company

**Job Description**At Boeing, we innovate and collaborate to make the world a better place. We're committed to fostering an environment...  ...SE&I National Team is a formal consortium of major aerospace companies aligned in partnership to provide the Missile Defense Agency (... 

Cabela's

Seasonal Apparel Stocking Associate Job at Cabela's

 ...reasonable accommodation under the Americans with Disabilities Act and certain state or local laws. If you need a reasonable accommodation for any part of the application process, please visit your nearest location or contact us at ****@*****.***. Cabela's