Senior AI Inference Infrastructure Engineer

Modular; United States; Hybrid; USD 167,000 - 273,000; Full time

Benefits offered by this job

Competitive salary

Premier insurance plans

Flexible paid time off

Team building events

Job summary

A leading AI infrastructure company is seeking a Software Engineer for its Inference Infrastructure team. This role involves building LLM-focused inference services and creating distributed systems for high performance AI solutions. The ideal candidate will possess over 5 years of backend engineering experience and a strong background in ML inference infrastructure. Candidates can work hybrid from Los Altos, CA or remotely, making a significant impact in a collaborative team. Competitive compensation and comprehensive benefits are offered.

Qualifications

• 5+ years of experience working in backend engineering.
• Experience working on high-scale ML inference infrastructure.
• Experience with Kubernetes and managing services.

Responsibilities

• Build and ship modular’s LLM focused inference services.
• Create distributed systems for high performance inference.
• Collaborate with kernels and genAI teams for application performance.

Skills

Backend engineering

Machine learning inference

Kubernetes

Problem-solving

Tools

vLLM

TensorRT-LLM

Golang

Job description

A leading AI infrastructure company is seeking a Software Engineer for its Inference Infrastructure team. This role involves building LLM-focused inference services and creating distributed systems for high performance AI solutions. The ideal candidate will possess over 5 years of backend engineering experience and a strong background in ML inference infrastructure. Candidates can work hybrid from Los Altos, CA or remotely, making a significant impact in a collaborative team. Competitive compensation and comprehensive benefits are offered.