Research Engineer, AI Systems & Distributed Training

Fireworks; San Mateo (CA); On-site; USD 140,000 - 200,000; Full time

Job summary

Fireworks is seeking a Research Engineer to join the team at the intersection of model research and training infrastructure. You will design novel architectures, reproduce state-of-the-art results, and build the distributed systems needed to scale experiments on large GPU clusters.

You’ll bridge science and engineering by translating theoretical ideas into robust, performant code, collaborating with researchers to push the envelope in enterprise AI infrastructure.

Qualifications

• Strong programming skills in Python, C++, or Rust.
• Deep practical knowledge of ML frameworks (PyTorch, JAX, or TensorFlow).
• Experience with large distributed systems and parallel computing (CUDA, NCCL, MPI).
• A strong foundation in linear algebra, calculus, probability, and statistics.
• A proven track record of implementing complex deep learning algorithms from scratch.

Responsibilities

• Conduct Open-Ended Research: Explore new model architectures, training objectives, and optimization techniques. Formulate hypotheses, design experiments, and iterate quickly based on empirical results.
• Reproduce and Extend State-of-the-Art: Implement and reproduce results from recent machine learning papers. Identify bottlenecks, propose improvements, and scale these methods to larger datasets and models.
• Build and Scale Training Infrastructure: Design, implement, and maintain high-performance, distributed machine learning systems. Optimize training loops, data loaders, and communication overhead across large GPU clusters.
• Bridge Science and Engineering: Translate abstract mathematical concepts and research ideas into robust, bug-free, and efficient code.
• Collaborate Cross-Functionally: Work closely with Research Scientists to unblock their experiments by providing tooling, optimizing code, and co-designing experiments that are hardware-aware.

Skills

Python

C++

Rust

Education

Master’s or PhD in CS/ML/Physics/Math

Tools

PyTorch

JAX

TensorFlow

Job description

Fireworks is seeking a Research Engineer to join the team at the intersection of model research and training infrastructure. You will design novel architectures, reproduce state-of-the-art results, and build the distributed systems needed to scale experiments on large GPU clusters.

You’ll bridge science and engineering by translating theoretical ideas into robust, performant code, collaborating with researchers to push the envelope in enterprise AI infrastructure.