AI Evaluation Platform Lead - Benchmarking & Insights

Thomson Reuters; Eagan (MN); On-site; USD 150,000 - 210,000; Full time

Job summary

Thomson Reuters Labs seeks a Lead Research Engineer to design and deliver production-ready software for AI model evaluation and benchmarking. You will build scalable evaluation infrastructure enabling teams to compare large language models, measure quality, cost, and latency, and guide model selection and development.

You will own the engineering approach from prototypes to reliable, maintainable tools, collaborating with researchers, engineers, and product partners while shaping technical

Qualifications

• 8-10+ years of relevant software engineering experience, including building and maintaining production systems involving ML or complex data.
• Strong hands-on Python programming skills, including API design, automated testing, and working with structured and unstructured data.
• Technical leadership and end-to-end delivery experience, including architecture decisions, engineering planning, staffing and effort estimation, project execution, and adapting priorities as requirements change.
• Experience collaborating with research scientists or ML practitioners to translate experiments and research methods into working software.
• Experience integrating or testing ML models, interpreting evaluation results, and applying findings to engineering decisions.
• Cloud engineering experience using AWS, Azure, or comparable platform, including distributed workflows, automated deployment, and monitoring.
• Working knowledge of SQL and experience developing software that processes structured and unstructured data.

Responsibilities

• Lead technical delivery and engineering planning with stakeholders to define priorities, scope, and staffing.
• Design and build scalable evaluation infrastructure, extending tools and delivering production-quality Python software and APIs.
• Turn research methods into working software by building automated evaluation pipelines and reproducible workflows.
• Maintain repeatable experiments with traceable results and clear links to inputs and methods.
• Build practical tools and interfaces for researchers and product teams including documentation and reports.
• Oversee production reliability, performance, and responsible data use with automated testing, deployment, and monitoring.

Skills

Python

API design

Automated testing

Cloud engineering

SQL

Mentoring

Team leadership

Data processing

Education

Bachelor's degree in CS/Engineering

Tools

AWS

Azure

APIs

Python

Job description

Thomson Reuters Labs seeks a Lead Research Engineer to design and deliver production-ready software for AI model evaluation and benchmarking. You will build scalable evaluation infrastructure enabling teams to compare large language models, measure quality, cost, and latency, and guide model selection and development.

You will own the engineering approach from prototypes to reliable, maintainable tools, collaborating with researchers, engineers, and product partners while shaping technical