Staff Engineer, Distributed ML Infrastructure

Fireworks AI; United States; On-site; USD 140,000 - 220,000; Full time

Job summary

Fireworks AI is hiring a Training Infrastructure Engineer to design, build, and maintain large-scale backend and cloud-native infrastructure for distributed ML training, inference, and data processing.

You will lead technical design discussions, mentor engineers, and drive best practices for scalable ML systems, focusing on reliability, low latency, and cost optimization. Proficiency with Docker, Kubernetes, and modern server-side languages is required.

Qualifications

• Design, develop, and optimize large-scale backend infrastructure and distributed data systems in cloud environments.
• Define and implement data-driven metrics to support company goals.
• Lead cross-functional projects and contribute to technical design documentation.
• Conduct coding interviews and provide structured feedback for engineering candidates.
• Collaborate with ML, DevOps, and product teams to translate requirements into infrastructure solutions.
• Experience with cloud-native tooling and infrastructure (Docker, Kubernetes).
• Proficiency in major server-side languages (Python, C++, Go, TypeScript) and data processing/API systems (gRPC/Thrift).

Responsibilities

• Design, build, and maintain scalable backend infrastructure for ML training, inference, and data processing pipelines.
• Architect scalable, resilient systems emphasizing reliability and low latency.
• Lead design discussions, mentor engineers, and establish best practices for ML infrastructure.
• Optimize compute, storage, and network performance to reduce costs and improve efficiency.
• Collaborate with ML, DevOps, and product teams to convert research requirements into robust infra.
• Evaluate and integrate cloud-native/open-source tech (Kubernetes, Ray, Kubeflow, MLFlow).
• Own end-to-end systems from design to deployment with fault tolerance and operations in mind.

Skills

Backend design

Distributed data systems

AB testing

Cross-functional collaboration

Mentoring engineers

Interviewing engineers

Education

Bachelor's degree in Computer Science or related field

Tools

Docker

Kubernetes

Python

Go

TypeScript

C++

PostgreSQL

MySQL

DynamoDB

Apache Spark

Apache Kafka

Apache Flink

Ray

Kubeflow

MLFlow

gRPC

Thrift

Job description

Fireworks AI is hiring a Training Infrastructure Engineer to design, build, and maintain large-scale backend and cloud-native infrastructure for distributed ML training, inference, and data processing.

You will lead technical design discussions, mentor engineers, and drive best practices for scalable ML systems, focusing on reliability, low latency, and cost optimization. Proficiency with Docker, Kubernetes, and modern server-side languages is required.