AI Infrastructure Engineer - Scalable ML Platform

Fireworks AI; San Mateo (CA); On-site; USD 175,000 - 220,000; Full time

Job summary

Fireworks AI is seeking a Software Engineer for our AI Infrastructure team to design core systems powering the generative platform. You will build scalable backend services, pipelines, and model serving components to ensure reliability and performance.

Collaborate with cloud, performance, and product teams to translate research into robust infrastructure, participate in CI/CD, and help advance open ML tooling and Kubernetes-based workflows.

Qualifications

• Bachelor's degree in CS or related field, or equivalent practical experience.
• 3 years in software engineering, with focus on infrastructure or ML systems.
• Strong Python/Go programming skills and solid coding practices.
• Experience with ML infra tooling (PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes).
• Basic understanding of LLM concepts such as context length and KV cache memory.

Responsibilities

• Contribute to scalable backend infrastructure for distributed training, inference, and data pipelines.
• Build and maintain core backend services like CI/CD, control plane, and model serving systems.
• Support performance, cost efficiency, and reliability across compute, storage, and networking layers.
• Collaborate with performance, training, and product teams to translate research into infrastructure solutions.
• Participate in code reviews and CI/CD processes.

Skills

Python

Go

ML infra

Kubernetes

Distributed systems

Performance

Education

Bachelor's degree in Computer Science or related field

Tools

PyTorch

MLflow

Vertex AI

SageMaker

Kubernetes

vLLM / SgLang / TRT-LLM

Job description

Fireworks AI is seeking a Software Engineer for our AI Infrastructure team to design core systems powering the generative platform. You will build scalable backend services, pipelines, and model serving components to ensure reliability and performance.

Collaborate with cloud, performance, and product teams to translate research into robust infrastructure, participate in CI/CD, and help advance open ML tooling and Kubernetes-based workflows.