Senior AI Evaluation & Benchmarking Lead

TRSS; Ann Arbor (MI); Hybrid; USD 127,000 - 237,000; Full time

Benefits offered by this job

Hybrid Work Model

Mental Health Days

Headspace app

Tuition Reimbursement

Employee incentive programs

Job summary

Thomson Reuters Labs seeks a Lead Research Engineer to design and deliver production-grade software for AI model evaluation. You will build scalable evaluation infrastructure to compare large language models, measure quality, cost, and latency, and guide model selection and development.

You will own the engineering approach from prototypes to maintainable tools, partnering with researchers, engineers, and product teams while shaping technical direction and delivery.

Qualifications

• 8–10+ years of software engineering experience
• Strong Python programming with API design and testing
• Technical leadership and end-to-end delivery experience
• Experience collaborating with researchers to translate experiments into software
• Experience integrating/testing machine learning models and evaluating results
• Cloud engineering experience with AWS/Azure and distributed workflows
• Knowledge of SQL and processing structured/unstructured data
• Excellent collaboration, communication, and mentoring skills
• Bachelor's degree in CS/Engineering or equivalent

Responsibilities

• Lead technical delivery and engineering planning with stakeholders
• Design and build scalable evaluation infrastructure and reusable tools
• Turn research methods into working software with automated evaluation pipelines
• Maintain repeatable experiments and traceability of results
• Build user interfaces, documentation, and diagnostic capabilities for researchers
• Ensure production reliability, performance, and data governance
• Mentor engineers and align delivery across research, engineering, and product teams

Skills

Python

Leadership

Research collaboration

Cloud engineering

SQL

Communication

Machine Learning

Project planning

Education

Bachelor's degree in Computer Science or Engineering

Tools

AWS

Azure

Job description

Thomson Reuters Labs seeks a Lead Research Engineer to design and deliver production-grade software for AI model evaluation. You will build scalable evaluation infrastructure to compare large language models, measure quality, cost, and latency, and guide model selection and development.

You will own the engineering approach from prototypes to maintainable tools, partnering with researchers, engineers, and product teams while shaping technical direction and delivery.