Senior Staff ML Engineer — GenAI Evaluation Lead (Remote)

Nubeero Limited

Northern (KY)

Hybrid

USD 244,000 - 305,000

Full time

2 days ago

Be an early applicant

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Airbnb is seeking a Senior Staff Machine Learning Engineer to set technical direction for ML evaluation and the data flywheel powering Airbnb Assistance Engineering. You will define metrics, build scalable evaluation systems, and partner across product, engineering, design, and operations to drive safe, impactful AI-powered experiences.

The role emphasizes governance, bias controls, and production-ready pipelines to enable continuous improvement of models and products.

Qualifications

PhD in Computer Science, Mathematics, Statistics, or related technical field (or equivalent practical experience).
10+ years building, testing, and shipping ML/AI systems end-to-end; including 2+ years with GenAI/LLM in production.
5+ years leading large, ambiguous technical initiatives as a senior IC.

Responsibilities

Define evaluation strategy and success metrics for GenAI systems, aligning offline evaluation with online business outcomes.
Build and scale evaluation frameworks with controls for bias, drift, and reliability.
Design the data flywheel with instrumentation, labeling, versioning, and governance to support continuous improvement.
Lead cross-functional quality initiatives across product, ops, and engineering.