Senior Staff ML Engineer: GenAI Evaluation & Data Flywheel

Airbnb; Gunnison (CO); On-site; USD 150,000 - 210,000; Full time

Job summary

Airbnb is seeking a Senior Staff Machine Learning Engineer to set technical direction for evaluation of GenAI systems and to lead end-to-end data flywheel powering Airbnb Assistance Engineering like assistive agents and issue resolution. You will define evaluation criteria, ensure alignment with online outcomes, and partner with product, design, and operations to improve model safety and usefulness.

The role emphasizes building scalable evaluation platforms, governance, and production-ready

Qualifications

• PhD in Computer Science, Mathematics, Statistics, or related field (or equivalent practical experience).
• 10+ years building, testing, and shipping ML/AI systems end-to-end; 2+ years GenAI/LLM in production.
• 5+ years leading large, ambiguous technical initiatives as a senior IC.
• Offline/online evaluation, metrics design, human-in-the-loop evaluation, A/B testing, regression testing.

Responsibilities

• Define evaluation strategy and success metrics for GenAI systems.
• Build and scale evaluation frameworks with bias controls and reliability.
• Design the data flywheel: instrumentation, labeling, versioning, governance.
• Lead cross-functional quality initiatives across product, ops, and engineering.
• Develop and productionize pipelines for dataset creation, model monitoring, and continuous testing.

Skills

Evaluation methodology

GenAI/LLM systems production

Data pipelines & quality systems

ML fundamentals

Education

PhD in CS/Math/Stats or related field

Job description

Airbnb is seeking a Senior Staff Machine Learning Engineer to set technical direction for evaluation of GenAI systems and to lead end-to-end data flywheel powering Airbnb Assistance Engineering like assistive agents and issue resolution. You will define evaluation criteria, ensure alignment with online outcomes, and partner with product, design, and operations to improve model safety and usefulness.

The role emphasizes building scalable evaluation platforms, governance, and production-ready