LLM Infrastructure Engineer — Performance & Scale

Fireworks AI; United States; On-site; USD 150,000 - 210,000; Full time

Job summary

Fireworks AI is seeking a Software Engineer for the AI Infrastructure team to design core systems powering our generative platform. You will help build infrastructure and tools for reliability, performance, and availability of the AI system.

You will partner with the cloud, product, and performance teams to deliver infrastructure that connects customers to our ultra-performant inference engine, supporting distributed training, inference, and data pipelines.

Qualifications

• 3 years of software engineering experience with infrastructure or ML systems.
• 5+ years of software engineering experience with infrastructure or machine learning systems.
• Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes).
• Experience with open source inference engines like vLLM, Sglang, or TRT-LLM.
• Contributions to open-source infrastructure or ML projects.

Responsibilities

• Design core systems powering Fireworks AI's generative platform.
• Build infrastructure and tools ensuring reliability, performance, quality and availability.
• Collaborate with cloud, product, and performance teams to bridge customers and Fireworks engine.
• Develop scalable backend infra for distributed training, inference, and data pipelines.
• Build and maintain core backend services like LLM CI/CD, control plane, and model serving.

Skills

Python

Go

ML infra tooling

Kubernetes

PyTorch

MLflow

Vertex AI

SageMaker

Open source infra

LLM basics

Education

Bachelor's degree in Computer Science or related field

Tools

Open source inference engines

Job description

Fireworks AI is seeking a Software Engineer for the AI Infrastructure team to design core systems powering our generative platform. You will help build infrastructure and tools for reliability, performance, and availability of the AI system.

You will partner with the cloud, product, and performance teams to deliver infrastructure that connects customers to our ultra-performant inference engine, supporting distributed training, inference, and data pipelines.