Staff LLM Infrastructure Engineer — Scalable AI Backend

Fdetimes; San Mateo (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

Fireworks is seeking a Software Engineer for our AI Infrastructure team to design and build the core systems powering our generative AI platform. You will help create scalable backend services for distributed training, inference, and data pipelines, ensuring reliability and performance.

You will collaborate with cloud, product, and performance teams to translate research needs into infrastructure that supports enterprise AI at scale, with a focus on quality and cost efficiency.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related technical field (or equivalent practical experience).
• 3 years of experience in software engineering, with a focus on infrastructure or machine learning systems.
• Strong programming skills in Python, Go, or a similar language.
• Proven experience in ML infrastructure and tooling (e.g., PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes, etc.).
• Basic understanding of LLM knowledge (e.g., context length, disaggregated prefill, KV cache memory estimation, etc)

Responsibilities

• Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines.
• Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems.
• Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers.
• Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry.
• Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions.
• Participate in code reviews, technical discussions, and continuous integration and deployment processes

Skills

Python

Go

Distributed systems

Education

Bachelor's degree in Computer Science or related field

Tools

PyTorch

MLflow

Kubernetes

Vertex AI

SageMaker

Job description

Fireworks is seeking a Software Engineer for our AI Infrastructure team to design and build the core systems powering our generative AI platform. You will help create scalable backend services for distributed training, inference, and data pipelines, ensuring reliability and performance.

You will collaborate with cloud, product, and performance teams to translate research needs into infrastructure that supports enterprise AI at scale, with a focus on quality and cost efficiency.