Senior Staff ML Engineer, GenAI Evaluation & Data Flywheel

Airbnb, Inc.

Oregon

Hybrid

USD 244,000 - 305,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Airbnb, Inc. is seeking a Senior Staff Machine Learning Engineer to lead evaluation on GenAI-powered services. You will define evaluation strategies, align offline metrics with online outcomes, and shape a scalable data flywheel powering Airbnb Assistance Engineering.

You will collaborate with product, engineering, design, and operations to build trusted evaluation systems, monitor model quality, implement guardrails, and drive continuous learning in a fast-paced, globally deployed platform.

Qualifications

  • 10+ years building, testing, and shipping ML/AI end-to-end systems, with 2+ years GenAI/LLM in production.
  • 5+ years leading large, ambiguous technical initiatives as a senior IC, influencing roadmap across teams.
  • Deep expertise in evaluation methodology (offline/online alignment, metric design, A/B testing, power analysis).
  • Hands-on GenAI experience including orchestration, retrieval, tool calling, memory.
  • Experience building data pipelines and quality systems (labeling, dataset versioning, monitoring, governance).
  • Solid ML fundamentals: model selection, training/serving, monitoring, lifecycle management.
  • Ability to collaborate with product, engineering, design, operations to build scalable evaluation systems.

Responsibilities

  • Define evaluation strategy and success metrics for GenAI systems, aligning offline evaluation with online outcomes.
  • Build and scale evaluation frameworks with controls for bias, drift, reliability.
  • Design the data flywheel: instrumentation, feedback collection, data quality checks, labeling strategy, versioning, governance.
  • Lead cross-functional quality initiatives across product, ops, and engineering.
  • Develop and productionize pipelines for dataset creation, model monitoring, evaluation-at-scale, and continuous testing.
  • Drive technical decisions and architecture for evaluation and data infrastructure.

Education

PhD in CS/Math/Stats

Job description

Airbnb, Inc. is seeking a Senior Staff Machine Learning Engineer to lead evaluation on GenAI-powered services. You will define evaluation strategies, align offline metrics with online outcomes, and shape a scalable data flywheel powering Airbnb Assistance Engineering.

You will collaborate with product, engineering, design, and operations to build trusted evaluation systems, monitor model quality, implement guardrails, and drive continuous learning in a fast-paced, globally deployed platform.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Staff Machine Learning Engineer, Data & Eval
Senior Staff Machine Learning Engineer, Data & Eval

Airbnb, Inc. • United States

Remote
USD 150,000 - 210,000
Bonus
Equity
Employee Travel Credits
A Senior Staff Machine Learning Engineer, Data & Eval Airbnb via Greenhouse Software engineering View role
A Senior Staff Machine Learning Engineer, Data & Eval Airbnb via Greenhouse Software engineering View role

Nubeero Limited • United States

Hybrid
USD 244,000 - 305,000
Senior Staff Machine Learning Engineer, Data & Eval
Senior Staff Machine Learning Engineer, Data & Eval

Airbnb, Inc. • Oregon

Hybrid
USD 244,000 - 305,000
Senior Staff ML Engineer - LLM Fine-Tuning & AI Platform
Senior Staff ML Engineer - LLM Fine-Tuning & AI Platform

Airbnb, Inc. • Oregon

Hybrid
USD 248,000 - 310,000
Bonus
Equity
Employee Travel Credits
+1
Staff ML Engineer Generative AI for Customer Care (Remote)
Staff ML Engineer Generative AI for Customer Care (Remote)

Airbnb, Inc. • United States

Remote
USD 212,000 - 260,000
Senior Staff ML Engineer — GenAI Evaluation Lead (Remote)
Senior Staff ML Engineer — GenAI Evaluation Lead (Remote)

Nubeero Limited • United States

Hybrid
USD 244,000 - 305,000
Senior ML Engineer, Trust AI & Autonomous Agents
Senior ML Engineer, Trust AI & Autonomous Agents

Airbnb, Inc. • United States

On-site
USD 150,000 - 230,000
Staff ML Engineer - AI for Customer Support (Remote)
Staff ML Engineer - AI for Customer Support (Remote)

Airbnb, Inc. • San Francisco (CA)

Remote
USD 212,000 - 260,000
Bonus opportunities
Employee travel credits
Equity options
+1
Staff ML Engineer – Generative AI for Customer Support
Staff ML Engineer – Generative AI for Customer Support

airbnb, Inc. • San Francisco (CA)

On-site
USD 140,000 - 180,000
Employee Travel Credits
Bonus and equity opportunities
Inclusive culture
Senior ML Engineer, Trust & Fraud AI Leader
Senior ML Engineer, Trust & Fraud AI Leader

Airbnb, Inc. • United States

Remote
USD 244,000 - 305,000