Senior LLM Infrastructure Engineer

Fireworks AI, Inc.; San Mateo (CA); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Fireworks AI, Inc. is seeking a Software Engineer for the AI Infrastructure team to help design core systems powering the generative AI platform.

You will build reliable infrastructure that ensures performance, quality, and availability across training, inference, and data pipelines. You will collaborate with cloud, product, and performance teams to translate research and product needs into robust infrastructure solutions and contribute to CI/CD and deployment processes for scalable model

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent practical experience).
• 3 years of software engineering experience with infrastructure or ML systems.
• Strong programming in Python or Go.
• Experience in ML infrastructure and tooling (PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes).
• Basic understanding of LLM knowledge (context length, KV cache memory, etc).

Responsibilities

• Contribute to the design and development of scalable backend infrastructure that supports distributed training, inference, and data pipelines
• Build and maintain core backend services such as LLM CI/CD pipeline, control plane, and model serving systems
• Support performance optimization, cost efficiency, and reliability improvements across compute, storage, and networking layers
• Building frameworks and safeguards to ensure Fireworks AI has the best model quality in the industry
• Collaborate with performance, training, and product teams to translate research and product needs into infrastructure solutions
• Participate in code reviews, technical discussions, and continuous integration and deployment processes

Skills

Python

Go

ML infra

Education

Bachelor’s degree in CS/Engineering

Tools

PyTorch

MLflow

Vertex AI

SageMaker

Kubernetes

Job description

Fireworks AI, Inc. is seeking a Software Engineer for the AI Infrastructure team to help design core systems powering the generative AI platform.

You will build reliable infrastructure that ensures performance, quality, and availability across training, inference, and data pipelines. You will collaborate with cloud, product, and performance teams to translate research and product needs into robust infrastructure solutions and contribute to CI/CD and deployment processes for scalable model