Software Engineer, AI Infrastructure & ML Systems

Fireworks AI; New York (NY); On-site; USD 175,000 - 220,000; Full time

Benefits offered by this job

Equity

Competitive salary

Comprehensive benefits

Job summary

Fireworks AI is hiring a Software Engineer for its AI Infrastructure team to design and build core systems powering our generative AI platform. You will collaborate with cloud, product, and performance teams to deliver scalable infrastructure for training, inference, and data pipelines.

The role focuses on reliability, performance, and availability, with opportunities to influence architecture and tooling across the stack and to contribute to cutting‑edge ML infrastructure and open‑source

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent practical experience)
• 3 years of experience in software engineering, with a focus on infrastructure or machine learning systems
• Strong programming skills in Python, Go, or a similar language
• Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, etc.)
• Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation)

Responsibilities

• Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines
• Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems
• Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers
• Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry
• Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions
• Participate in code reviews, technical discussions, and continuous integration and deployment processes

Skills

Python

Go

ML infrastructure

LLM knowledge

Education

Bachelor’s degree in CS/Engineering or related field

Tools

PyTorch

MLflow

Vertex AI

SageMaker

Kubernetes

vLLM

Sglang

TRT-LLM

Job description

Fireworks AI is hiring a Software Engineer for its AI Infrastructure team to design and build core systems powering our generative AI platform. You will collaborate with cloud, product, and performance teams to deliver scalable infrastructure for training, inference, and data pipelines.

The role focuses on reliability, performance, and availability, with opportunities to influence architecture and tooling across the stack and to contribute to cutting‑edge ML infrastructure and open‑source