Staff LLM Inference & Serving Engineer - Remote

Prime Intellect, Inc.; United States; Remote; USD 150,000 - 300,000; Full time

Benefits offered by this job

Remote work option

Visa sponsorship

Professional development budget

Team off-sites

Equity incentives

Job summary

Prime Intellect is assembling an open frontier AI stack and seeks engineers to advance LLM serving and RL integration at scale. The role blends cloud serving, inference optimization, and distributed systems across GPU fleets, with emphasis on performance, reliability, and extensibility.

You will build multi-tenant serving platforms, improve scheduling, and contribute to open infrastructure projects while collaborating with research and product teams in a fast-moving environment.

Qualifications

• 3+ years building and running large-scale ML/LLM services with latency/availability SLOs.
• Hands-on with at least one of vLLM, SGLang, or TensorRT-LLM.
• Experience with distributed serving infra (e.g., NVIDIA Dynamo).

Responsibilities

• Build multi-tenant LLM serving platform across cloud GPU fleets.
• Design GPU-aware scheduling and autoscaling to meet SLOs.
• Integrate inference into RL training stack and optimize performance.
• Develop observability, CI/CD, and documentation for scalable infra.

Skills

ML Systems at Scale

LLM Inference

Distributed Serving Infra

CUDA/NCCL Debugging

Python

PyTorch

Tools

vLLM

SGLang

TensorRT-LLM

NVIDIA Dynamo

Kubernetes

CUDA

Job description

Prime Intellect is assembling an open frontier AI stack and seeks engineers to advance LLM serving and RL integration at scale. The role blends cloud serving, inference optimization, and distributed systems across GPU fleets, with emphasis on performance, reliability, and extensibility.

You will build multi-tenant serving platforms, improve scheduling, and contribute to open infrastructure projects while collaborating with research and product teams in a fast-moving environment.