Member of Technical Staff

Fireworks AI; New York (NY); On-site; USD 170,000 - 260,000; Full time

Job summary

Fireworks AI in New York seeks a Training Infrastructure Engineer to design and maintain large-scale backend and cloud-native systems for distributed ML training and inference. You will architect scalable services, lead design discussions, and mentor engineers to improve reliability and performance across the platform.

You will collaborate with ML, DevOps, and product teams, evaluate Kubernetes, Ray, Kubeflow, and MLFlow, and own end-to-end systems from design to deployment with emphasis on

Qualifications

• Bachelor's degree in CS or related field; four+ years of backend infra experience.
• Experience building large-scale distributed data systems in cloud environments.
• Proficient in Python, C++, Go, or TS; strong design documentation skills.

Responsibilities

• Architect scalable backend services for training/inference platforms.
• Lead design discussions and mentor engineers across teams.
• Improve compute cost, storage, and network performance.
• Collaborate with ML, DevOps, and product to translate requirements.
• Evaluate cloud-native tools: Kubernetes, Ray, Kubeflow, MLFlow.

Skills

Backend design

Cloud services

Python

Go

TypeScript

C++

Leadership

Data pipelines

Education

Bachelor's degree in CS or related

Tools

Docker

Kubernetes

Ray

Kubeflow

MLFlow

PostgreSQL

Kafka

Job description

About Us

Fireworks is the platform for specialized intelligence, enabling companies to build, train, and serve AI models tailored to their own data, workflows, and products. Founded by the team behind PyTorch and backed by AMD, Atreides, Benchmark Capital, Index Ventures, Lightspeed, NVIDIA, Sequoia Capital, and TCV, Fireworks powers production AI with hundreds of state-of-the-art open models across text, image, embedding, audio, and multimodal workloads. Today, Fireworks is a Series D company valued at $17.5 billion, bringing together an ambitious, collaborative team that's building the future of enterprise AI.

About Us

Fireworks is the platform for specialized intelligence, enabling companies to build, train, and serve AI models tailored to their own data, workflows, and products. Founded by the team behind PyTorch and backed by AMD, Atreides, Benchmark Capital, Index Ventures, Lightspeed, NVIDIA, Sequoia Capital, and TCV, Fireworks powers production AI with hundreds of state-of-the-art open models across text, image, embedding, audio, and multimodal workloads. Today, Fireworks is a Series D company valued at $17.5 billion, bringing together an ambitious, collaborative team that's building the future of enterprise AI.

The Role

As a Training Infrastructure Engineer, you'll design, develop, and maintain large-scale backend and cloud-native infrastructure to support distributed machine learning training, inference, and data processing pipelines for our generative AI platform. You'll architect scalable, resilient backend infrastructure, lead technical design discussions, mentor engineers, and establish best practices for large-scale machine learning systems.

Key Responsibilities

• Architect and build scalable, resilient backend infrastructure to support distributed training, inference, and data processing pipelines
• Lead technical design discussions, mentor engineers, and establish best practices for large-scale machine learning systems
• Design and implement core backend services with a focus on efficiency and low latency
• Drive infrastructure optimization initiatives for compute cost, storage lifecycle management, and network performance
• Collaborate with machine learning, DevOps, and product teams to translate research and product requirements into robust infrastructure solutions
• Evaluate and integrate cloud-native and open-source technologies such as Kubernetes, Ray, Kubeflow, and MLFlow to enhance platform reliability
• Own end-to-end systems from design to deployment, emphasizing reliability, fault tolerance, and operational excellence

Minimum Qualifications

• Bachelor's degree or equivalent in Computer Science or related field plus four (4) years of experience in software engineering or related role
• 4 years of experience designing, building, and optimizing large-scale backend infrastructure and distributed data systems (e.g., PostgreSQL, MySQL, DynamoDB, Apache Spark, Apache Flink, Apache Kafka) in cloud environments (AWS, GCP, Azure, or equivalent), including cloud-native platforms, core infrastructure components, and optimization techniques (caching, indexing, sharding, replication, transactions, ACID)
• 4 years of experience with major server-side programming languages and frameworks (e.g., Python, C++, Go, TypeScript)
• 4 years of experience writing technical design documentation, leading cross-functional projects, and collaborating with cross-functional teams to achieve business impact
• 3 years of experience developing and maintaining data processing and API systems, including client-server communication frameworks (e.g., gRPC, Thrift)
• 3 years of experience conducting A/B testing and scientific experimentation (e.g., Statsig, Meta Deltoid, Optimizely) to measure software impact
• 3 years of experience conducting coding interviews and providing systematic feedback for engineering candidates
• 2 years of experience with cloud-native tools and infrastructure, such as Docker and Kubernetes
• 2 years of experience defining and implementing data-driven metrics to support company or team goals

Why Fireworks?

• Solve Hard Problems: Tackle challenges at the forefront of AI infrastructure, from low-latency inference to scalable model serving.
• Build What’s Next: Work with bleeding-edge technology that impacts how businesses and developers harness AI globally.
• Ownership & Impact: Join a fast-growing, passionate team where your work directly shapes the future of AI- no bureaucracy, just results.
• Learn from the Best: Collaborate with world-class engineers and AI researchers who thrive on curiosity and innovation.

Fireworks AI is an equal-opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all innovators.