Staff Cloud Infrastructure Engineer – ML/AI Platform

Fireworks AI, Inc.

San Mateo (CA)

Hybrid

USD 175,000 - 220,000

Full time

7 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity
Hybrid work model

Job summary

Fireworks AI seeks a highly skilled Software Engineer to join the Cloud Infrastructure team in building the platform that powers AI workloads globally. You’ll architect scalable backend systems for distributed training, inference, and data processing across multiple cloud providers.

In this role you’ll mentor peers, design core services like schedulers and autoscalers, and push for low-latency, cost-efficient solutions.

Qualifications

  • Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent practical experience).
  • 5+ years designing and building backend infrastructure in cloud environments (e.g., AWS, GCP, Azure).
  • Proven experience in ML infrastructure and tooling (e.g., PyTorch, TensorFlow, Vertex AI, SageMaker, Kubernetes, etc.).
  • Strong software development skills in languages like Python, or C++.
  • Deep understanding of distributed systems fundamentals: scheduling, orchestration, storage, networking, and compute optimization.

Responsibilities

  • Architect and build scalable backend infrastructure to support distributed training, inference, and data processing pipelines.
  • Lead technical design discussions, mentor other engineers, and establish best practices for building and operating large-scale ML infrastructure.
  • Design and implement core backend services (e.g., job schedulers, resource managers, autoscalers, model serving layers) with a focus on efficiency and low latency.
  • Drive infrastructure optimization initiatives, including compute cost reduction, storage lifecycle management, and network performance tuning.
  • Collaborate cross-functionally with ML, DevOps, and product teams to translate research and product needs into robust infrastructure solutions.
  • Continuously evaluate and integrate cloud-native and open-source technologies (e.g., Kubernetes, Kubeflow, MLFlow) to enhance our platform’s capabilities and reliability.
  • Own end-to-end systems from design to deployment and observability, with a strong emphasis on reliability, fault tolerance, and operational excellence.
  • Ensuring System Reliability: Ensure systems are designed and implemented with high availability, scalability, and performance.

Skills

Python
C++
Distributed systems
Cloud infrastructure
ML infrastructure
Kubernetes
Job scheduling
Observability
Performance optimization

Education

Bachelor's degree in Computer Science
Master's or PhD (preferred)

Tools

Terraform
ArgoCD
GitOps
Vertex AI
SageMaker
Kubeflow
MLFlow
AWS
GCP
Azure

Job description

Fireworks AI seeks a highly skilled Software Engineer to join the Cloud Infrastructure team in building the platform that powers AI workloads globally. You’ll architect scalable backend systems for distributed training, inference, and data processing across multiple cloud providers.

In this role you’ll mentor peers, design core services like schedulers and autoscalers, and push for low-latency, cost-efficient solutions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Cloud Infrastructure Engineer - Scalable ML Platform
Senior Cloud Infrastructure Engineer - Scalable ML Platform

Fireworks • New York (NY), San Mateo (CA)

On-site
USD 140,000 - 210,000
Staff LLM Infrastructure Engineer — Scalable AI Backend
Staff LLM Infrastructure Engineer — Scalable AI Backend

Fdetimes • San Mateo (CA)

On-site
USD 140,000 - 210,000
Member of Technical Staff, Cloud Infrastructure
Member of Technical Staff, Cloud Infrastructure

Fireworks • New York (NY), San Mateo (CA)

On-site
USD 140,000 - 210,000
Member of Technical Staff, Cloud Infrastructure job at Fireworks AI in San Mateo, CA, New York, NY
Member of Technical Staff, Cloud Infrastructure job at Fireworks AI in San Mateo, CA, New York, NY

Fireworks AI, Inc. • San Mateo (CA)

Hybrid
USD 175,000 - 220,000
Equity
Hybrid work model
Staff Engineer, Enterprise AI Infrastructure
Staff Engineer, Enterprise AI Infrastructure

Fireworks AI, Inc. • New York (NY)

On-site
USD 180,000 - 260,000
Staff Systems Architect, AI Infrastructure at Scale
Staff Systems Architect, AI Infrastructure at Scale

Fireworks • San Mateo (CA)

On-site
USD 180,000 - 240,000
Senior LLM Infrastructure Engineer
Senior LLM Infrastructure Engineer

Fireworks AI, Inc. • San Mateo (CA)

Hybrid
USD 180,000 - 240,000
Senior Backend Engineer, AI Infrastructure & Platform
Senior Backend Engineer, AI Infrastructure & Platform

Fireworks • San Mateo (CA)

On-site
USD 175,000 - 220,000
Equity
Competitive salary
Benefits package
Staff AI Infrastructure Engineer
Staff AI Infrastructure Engineer

Fireworks AI, Inc. • City of Syracuse (NY), City of Auburn (NY)

On-site
USD 120,000 - 180,000
Senior AI Training Infrastructure Engineer
Senior AI Training Infrastructure Engineer

Fireworks • San Francisco (CA)

On-site
USD 150,000 - 230,000