Staff Cloud Infrastructure Engineer — ML & Distributed Systems

Fireworks AI; New York (NY); On-site; USD 175,000 - 220,000; Full time

Job summary

Fireworks AI in New York is seeking a Software Engineer for Cloud Infrastructure to design and build scalable backend systems that power distributed AI workloads across multiple clouds worldwide. You will shape architecture, improve reliability, and drive performance in a fast-growing platform.

Join a team of engineers and researchers focused on enterprise AI, with ownership of critical infrastructure and opportunities to influence cost efficiency, reliability, and scale across compute, storage,

Qualifications

• 5+ years designing backend infra in cloud environments (AWS, GCP, Azure).
• Experience with ML infrastructure and tooling (PyTorch, TensorFlow, Kubernetes).
• Strong Python or C++ software development skills.
• Deep understanding of distributed systems fundamentals.

Responsibilities

• Architect scalable backend infra for distributed training and data pipelines.
• Lead design discussions, mentor engineers, and establish best practices for ML infrastructure.
• Design and implement core backend services (e.g., schedulers, resource managers, model serving) with focus on efficiency and low latency.
• Drive infrastructure optimization: compute cost reduction, storage lifecycle management, network performance tuning.
• Collaborate with ML, DevOps, and product teams to translate research and product needs into infrastructure solutions.
• Evaluate and integrate cloud-native/open-source technologies (Kubernetes, Kubeflow, MLFlow) to enhance reliability.
• Own end-to-end systems from design to deployment and observability with high availability and fault tolerance.

Skills

Python

C++

Distributed systems

ML infrastructure

Education

Bachelor's degree in CS/Engineering

Tools

Kubernetes

TensorFlow

PyTorch

Vertex AI

SageMaker

Terraform

ArgoCD

GitOps

Job description

Fireworks AI in New York is seeking a Software Engineer for Cloud Infrastructure to design and build scalable backend systems that power distributed AI workloads across multiple clouds worldwide. You will shape architecture, improve reliability, and drive performance in a fast-growing platform.

Join a team of engineers and researchers focused on enterprise AI, with ownership of critical infrastructure and opportunities to influence cost efficiency, reliability, and scale across compute, storage,