LLM Infrastructure Engineer - Scalable AI Platform

Fireworks AI; San Mateo (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

Fireworks AI is seeking a Software Engineer for its AI Infrastructure team to design core systems powering a generative AI platform. You will help build reliable, high-performance infrastructure and tools that support training, inference, and data pipelines.

You'll work with cloud, product, and performance teams to bridge customer needs with the ultra-performant inference engine, contributing to scalable backend services and robust ML workflows.

Qualifications

• 3 years of software engineering with a focus on infrastructure or ML systems.
• Bachelor's degree in CS, engineering, or equivalent practical experience.
• Strong programming skills in Python, Go, or similar languages.

Responsibilities

• Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines
• Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems
• Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers
• Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry
• Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions
• Participate in code reviews, technical discussions, and continuous integration and deployment processes

Skills

Python

Go

ML infrastructure

Education

Bachelor's degree in Computer Science, Engineering, or a related technical field

Tools

Kubernetes

MLflow

Vertex AI

SageMaker

PyTorch

Job description

Fireworks AI is seeking a Software Engineer for its AI Infrastructure team to design core systems powering a generative AI platform. You will help build reliable, high-performance infrastructure and tools that support training, inference, and data pipelines.

You'll work with cloud, product, and performance teams to bridge customer needs with the ultra-performant inference engine, contributing to scalable backend services and robust ML workflows.