Staff Engineer, Evals & Post-Training Platform

Fireworks; San Mateo (CA); Presencial; USD 150 000 - 210 000; Tempo integral

Resumo da oferta

Fireworks is seeking a Member of Technical Staff, Evals & Post-Training Product to define efficient model evaluation and post-training workflows. You will bridge backend systems, sandbox infra, and user surfaces to simplify authoring evals, understanding results, and rapid iteration.

You will own scalable eval infrastructure, reproduce state-of-the-art benchmarks, and help design new benchmarks for domain-specific use cases.

Qualificações

• 1–7 years of software eng or data science experience.
• Strong system design for scalable, programmatic infrastructure.
• Hands-on with sandbox environments for secure code execution and testing.
• Deep understanding of LLM evaluations and how results guide model improvement.
• Familiarity with end-to-end GenAI lifecycle, from prompting to fine-tuning and deployment.

Responsabilidades

• Scale evaluation infrastructure and remove single-host bottlenecks.
• Reproduce SOTA benchmarks and ensure rigorous results.
• Design and build new benchmarks for complex or domain-specific use cases.
• Own the fine-tuning product experiences for SFT/RFT-related workflows.
• Collaborate with customers and internal teams to convert bespoke workflows into reusable solutions.

Conhecimentos

System design

Sandbox infrastructure

LLM evaluation

Data science mindset

End-to-end AI lifecycle

Ferramentas

Harbor framework

Descrição da oferta de emprego

Fireworks is seeking a Member of Technical Staff, Evals & Post-Training Product to define efficient model evaluation and post-training workflows. You will bridge backend systems, sandbox infra, and user surfaces to simplify authoring evals, understanding results, and rapid iteration.

You will own scalable eval infrastructure, reproduce state-of-the-art benchmarks, and help design new benchmarks for domain-specific use cases.