Senior AI Evaluation & Benchmarking Lead (Hybrid)

Thomson Reuters; New York (NY); Hybrid; USD 127,000 - 273,000; Full time

Job summary

Thomson Reuters Labs is seeking a Lead Research Engineer to design and deliver production-ready software for AI model evaluation and benchmarking. You will build scalable evaluation infrastructure to compare LLMs, measure quality, cost, and latency, and guide model selection and development with researchers, engineers, and product partners.

Responsibilities include leading planning, extending tools, and turning research methods into reliable software.

Qualifications

• 8-10+ years of software engineering with production ML experiences.
• Hands-on Python programming, API design, testing, and data handling.
• Leadership and end-to-end delivery across research, engineering and product.
• Experience translating experiments into deployable software.
• Knowledge of language models and ML evaluation.
• Cloud platforms (AWS/Azure) and distributed workflows.
• SQL and data-processing for structured/unstructured data.

Responsibilities

• Lead technical delivery and engineering planning across teams.
• Design scalable evaluation infra and production-grade Python tools.
• Build automated evaluation pipelines and reusable frameworks.
• Maintain repeatable experiments with traceable results.
• Collaborate with researchers to translate experiments to software.
• Ensure reliability, performance, and responsible data use.
• Mentor engineers and coordinate delivery with product partners.

Skills

Software engineering

Python

Technical leadership

ML model evaluation

Experimentation

Cloud engineering

SQL

Collaboration

Education

Bachelor's degree in CS/Engineering

Tools

AWS

Azure

Job description

Thomson Reuters Labs is seeking a Lead Research Engineer to design and deliver production-ready software for AI model evaluation and benchmarking. You will build scalable evaluation infrastructure to compare LLMs, measure quality, cost, and latency, and guide model selection and development with researchers, engineers, and product partners.

Responsibilities include leading planning, extending tools, and turning research methods into reliable software.