Research Engineer - AI Evaluation & Benchmarking

Uber AI Solutions; United States; Presencial; USD 150.000 - 190.000; Jornada completa

Descripción de la vacante

Uber AI Solutions seeks a Research Engineer to design, implement, and validate complex benchmark challenges that simulate real-world Research Engineering workflows. You will curate tasks requiring 1–2 days of focused effort, testing autonomous planning, code execution, and empirical research logic.

Daily collaboration with researchers ensures rigorous evaluation methods and reproducible results. The role emphasizes multi-step tasks, task guidelines, ground-truth references, and formal benchmark

Formación

• MSc or PhD in Computer Science, ML, AI, Data Science, Mathematics, Statistics, Physics, or related STEM field; equivalent practical experience considered.
• 1+ years of hands-on experience in Research Engineering, Machine Learning, Data Science, or computational AI research.
• Proven track record executing research projects involving coding, quantitative analysis, and rigorous experimental workflows.

Responsabilidades

• Benchmark Architecture: Design, curate, and review multi-step Research Engineering challenges that stress-test model reasoning and autonomous problem solving.
• Agentic Environments: Build realistic evaluation tasks requiring end-to-end planning, quantitative reasoning, code execution, experimentation, and active debugging.
• Technical Validation & QA: Rigorously verify tasks to ensure absolute technical accuracy, reproducible environments, and calibrated difficulty levels.
• Model Analysis: Evaluate model outputs across complex workflows to identify failure modes, edge cases, and unexpected model behavior.
• Iterative Research Collaboration: Participate in daily feedback sessions with research teams to refine evaluation methodologies, task specifications, and rubrics.
• Documentation: Author clear task guidelines, ground-truth references, and formal benchmark documentation.

Conocimientos

Python

Git workflows

ML & LLM literacy

Data analysis

Scientific methodology

Educación

MSc/PhD in Computer Science / ML / AI / Data Science / Math / Physics

Herramientas

Cursor

Gemini

Copilot

JetBrains AI

Descripción del empleo

Uber AI Solutions seeks a Research Engineer to design, implement, and validate complex benchmark challenges that simulate real-world Research Engineering workflows. You will curate tasks requiring 1–2 days of focused effort, testing autonomous planning, code execution, and empirical research logic.

Daily collaboration with researchers ensures rigorous evaluation methods and reproducible results. The role emphasizes multi-step tasks, task guidelines, ground-truth references, and formal benchmark