Staff Cloud Infrastructure Engineer, ML & AI Platforms

Fireworks AI; San Mateo (CA); On-site; USD 175,000 - 220,000; Full time

Job summary

Fireworks AI in California is seeking a Senior Software Engineer to lead our Cloud Infrastructure team, architecting scalable backend systems powering distributed training, inference, and data pipelines across multiple clouds. You’ll balance performance, cost, and reliability while delivering robust foundations for our platform.

This highly technical role requires deep expertise in distributed systems, cloud-native infrastructure, and ML platforms.

Qualifications

• BS in Computer Science, Engineering, or related field or equivalent practical experience.
• 5+ years designing and building backend infrastructure in cloud environments (AWS, GCP, Azure).
• Experience in ML infrastructure and tooling (PyTorch, TensorFlow, Vertex AI, SageMaker, Kubernetes).
• Strong software development in Python or C++.
• Deep understanding of distributed systems: scheduling, orchestration, storage, networking, compute optimization.

Responsibilities

• Architect and build scalable back-end infrastructure for distributed training, inference, and data processing pipelines.
• Lead technical design discussions, mentor other engineers, establish best practices for ML infrastructure.
• Design and implement core back-end services (job schedulers, resource managers, autoscalers, model serving) with focus on efficiency and low latency.
• Drive infra optimization: compute cost reduction, storage lifecycle management, and network performance tuning.
• Collaborate with ML, DevOps, and product teams to translate needs into robust infrastructure solutions.
• Continuously evaluate and integrate cloud-native/open-source technologies (Kubernetes, Kubeflow, MLFlow).
• Own end-to-end systems from design to deployment and observability, with reliability in mind.
• Ensure high availability, scalability, fault tolerance, and performance across multi-cloud infra.

Skills

Backend infrastructure

Cloud environments AWS/GCP/Azure

ML infrastructure

Python

C++

Distributed systems

Kubernetes

Education

Bachelor's degree in CS or related field

Master's or PhD in CS or related field

Tools

PyTorch

TensorFlow

Kubeflow

MLFlow

Terraform

Job description

Fireworks AI in California is seeking a Senior Software Engineer to lead our Cloud Infrastructure team, architecting scalable backend systems powering distributed training, inference, and data pipelines across multiple clouds. You’ll balance performance, cost, and reliability while delivering robust foundations for our platform.

This highly technical role requires deep expertise in distributed systems, cloud-native infrastructure, and ML platforms.