Senior LLM Infra Engineer — AI Model Serving

Nvidia Corporation; Santa Clara (CA); On-site; USD 184,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits package

Competitive salary

Job summary

NVIDIA in Santa Clara, CA, is hiring for a senior role focused on LLM serving infrastructure and platform API development. You will own the fine-tuning pipeline, evaluation harness, and observability layers while coordinating with ISVs and CSPs to deploy NVIDIA NIMs at scale.

Required: 7+ years in LLM serving infra, deep quantization knowledge, and strong collaboration across teams. A competitive salary and equity benefit package accompany this opportunity.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or equivalent experience.
• 7+ years of experience with LLM serving infrastructure (vLLM, TRT-LLM, or equivalent).
• Deep understanding of model quantization and fine-tuning workflows.
• Demonstrated history of developing robust platform APIs.
• Strong collaborative skills and experience working in cross-functional teams.
• Outstanding problem-solving skills and meticulous attention to detail.

Responsibilities

• Compose and build the fine-tuning handoff pipeline, including LoRA adapter repackaging, re-quantization, and re-validation into NIM.
• Develop the evaluation harness, ensuring models meet our high standards.
• Implement the observability and attestation layer to produce auditable compliance artifacts.
• Work in close partnership with ISVs and CSPs to roll out NVIDIA NIMs on a large scale.
• Define and improve durable platform APIs, steering clear of one-off integrations.
• Ensure flawless completion of projects through strict attention to detail and proven methodologies.

Skills

LLM serving infra

Model quantization

API development

Cross-functional collaboration

Problem solving

Education

Bachelor's degree in CS/engineering or equivalent

Tools

VLLM

TRT-LLM

Job description

NVIDIA in Santa Clara, CA, is hiring for a senior role focused on LLM serving infrastructure and platform API development. You will own the fine-tuning pipeline, evaluation harness, and observability layers while coordinating with ISVs and CSPs to deploy NVIDIA NIMs at scale.

Required: 7+ years in LLM serving infra, deep quantization knowledge, and strong collaboration across teams. A competitive salary and equity benefit package accompany this opportunity.