Research Engineer – AI Evaluation & Benchmarks (Freelance)

Uber AI Solutions; United States; On-site; USD 150,000 - 190,000; Full time

Job summary

Uber AI Solutions seeks a Research Engineer to design, implement, and validate complex benchmark challenges that simulate real-world Research Engineering workflows. You will curate tasks requiring 1–2 days of focused effort, testing autonomous planning, code execution, and empirical research logic.

Daily collaboration with researchers ensures rigorous evaluation methods and reproducible results. The role emphasizes multi-step tasks, task guidelines, ground-truth references, and formal benchmark

Qualifications

• MSc or PhD in Computer Science, ML, AI, Data Science, Mathematics, Statistics, Physics, or related STEM field; equivalent practical experience considered.
• 1+ years of hands-on experience in Research Engineering, Machine Learning, Data Science, or computational AI research.
• Proven track record executing research projects involving coding, quantitative analysis, and rigorous experimental workflows.

Responsibilities

• Benchmark Architecture: Design, curate, and review multi-step Research Engineering challenges that stress-test model reasoning and autonomous problem solving.
• Agentic Environments: Build realistic evaluation tasks requiring end-to-end planning, quantitative reasoning, code execution, experimentation, and active debugging.
• Technical Validation & QA: Rigorously verify tasks to ensure absolute technical accuracy, reproducible environments, and calibrated difficulty levels.
• Model Analysis: Evaluate model outputs across complex workflows to identify failure modes, edge cases, and unexpected model behavior.
• Iterative Research Collaboration: Participate in daily feedback sessions with research teams to refine evaluation methodologies, task specifications, and rubrics.
• Documentation: Author clear task guidelines, ground-truth references, and formal benchmark documentation.

Skills

Python

Git workflows

ML & LLM literacy

Data analysis

Scientific methodology

Education

MSc/PhD in Computer Science / ML / AI / Data Science / Math / Physics

Tools

Cursor

Gemini

Copilot

JetBrains AI

Job description

Uber AI Solutions connects accomplished industry professionals and researchers with leading Generative AI labs and enterprise product teams. We partner with top-tier technical experts to advance how frontier large language models (LLMs) understand, reason through, and solve complex, real-world engineering workflows.

About the Engagement

We are seeking a Research Engineer to design, implement, and validate complex, multi-step benchmark challenges that simulate real-world Research Engineering workflows. These scenarios will directly shape the next generation of rigorous evaluation suites used to assess and improve state-of-the-­art foundation models.

In this role, you will curate benchmark tasks that require 1–2 days of focused effort to solve—testing autonomous planning, code execution, and empirical research logic. You will work with experimental model checkpoints, advanced internal AI tooling, and tight, daily feedback loops with research scientists.

Key Responsibilities

• Benchmark Architecture: Design, curate, and review multi-step Research Engineering challenges that stress-test model reasoning and autonomous problem solving.
• Agentic Environments: Build realistic evaluation tasks requiring end-to-end planning, quantitative reasoning, code execution, experimentation, and active debugging.
• Technical Validation & QA: Rigorously verify tasks to ensure absolute technical accuracy, reproducible environments, and calibrated difficulty levels.
• Model Analysis: Evaluate model outputs across complex workflows to identify failure modes, edge cases, and unexpected model behavior.
• Iterative Research Collaboration: Participate in daily feedback sessions with research teams to refine evaluation methodologies, task specifications, and rubrics.
• Documentation: Author clear task guidelines, ground-truth references, and formal benchmark documentation.

Qualifications

Education & Background

• MSc or PhD in Computer Science, Machine Learning, Artificial Intelligence, Data Science, Mathematics, Statistics, Physics, or related STEM discipline (equivalent practical experience in computational research will also be considered).
• 1+ years of hands‑on experience in Research Engineering, Machine Learning, Data Science, or computational AI research.
• Demonstrated track record executing research projects involving coding, quantitative analysis, and rigorous experimental workflows.

Technical Capabilities

• Programming & Systems: Strong Python programming and scripting skills; experienced with modern Git workflows, debugging environments, and clean software engineering principles.
• ML & LLM Literacy: Solid grounding in machine learning fundamentals and experimental design. Working understanding of LLM capabilities, evaluation frameworks, and common failure patterns (RL exposure is a plus).
• Data & Empirical Analysis: Proficient in statistical analysis, exploratory data handling (Jupyter/Colab), and working with complex datasets.
• Scientific Rigor: Strong background in hypothesis testing, scientific methodology, and reproducible computational workflows.

Preferred Experience

• Prior work collaborating with Research Scientists or building AI evaluation benchmarks.
• Familiarity with agentic workflows, prompt engineering, red teaming, or AI safety.
• Daily power‑user experience with AI coding tools (e.g., Cursor, Gemini, Copilot, JetBrains AI).