AI Evaluation & Benchmarking Lead Engineer

Thomson Reuters; Ann Arbor (MI); Híbrido; USD 127.000 - 237.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Hybrid Work Model

Flex My Way / Work-Life Balance

Career Development and Growth

Industry Competitive Benefits

Descripción de la vacante

Thomson Reuters Labs seeks a Lead Research Engineer to design and deliver production-ready software for AI model evaluation and benchmarking. You will build scalable evaluation infrastructure that enables research and product teams to compare large language models, measure quality, cost, and latency, and help select models.

You will own the engineering approach from prototypes to reliable tools, collaborating with researchers, engineers, and product partners while shaping technical direction and

Formación

• 8-10+ years of relevant software engineering experience, including building and maintaining production systems involving machine learning or complex data.
• Strong hands-on Python programming skills, including API design, automated testing, and working with structured and unstructured data.
• Technical leadership and end-to-end delivery experience, including architecture decisions, engineering planning, staffing and effort estimation, project execution, and adapting priorities as requirements change.
• Experience collaborating with research scientists or machine learning practitioners to translate experiments and research methods into working software.
• Experience integrating or testing machine learning models, interpreting evaluation results, and applying findings to engineering decisions. Understanding of language models and the ability to recognize unreliable comparisons or test-data leakage.
• Cloud engineering experience using AWS, Azure, or a comparable platform, including distributed workflows, automated deployment, and monitoring.
• Working knowledge of SQL and experience developing software that processes structured and unstructured data.
• Strong collaboration, communication, problem-solving, and mentoring skills, with experience coordinating technical delivery across research, engineering, and product stakeholders.
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience.

Responsabilidades

• Lead technical delivery and engineering planning: Partner with program and research leaders to define priorities, break projects into delivery stages, estimate effort and cost, recommend staffing and required skills, identify dependencies, and adapt plans to available resources.
• Design and build scalable evaluation infrastructure: Extend existing shared tools and services, design new components where needed, and develop production-quality Python software, APIs, and reusable evaluation frameworks. Deliver useful initial versions, validate them with users, and improve them through iterative development.
• Turn research methods into working software: Collaborate with scientists to build automated evaluation pipelines connecting model APIs, test datasets, and agreed scoring methods. Implement reliable workflows for comparing LLMs and other AI models using evaluation datasets relevant to products and customer needs.
• Ensure reliable, reproducible model comparisons: Maintain repeatable experiments, separate evaluation data from model development, and make results traceable to evaluation inputs and methods. Measure model quality, operating cost, and latency to support product teams in validating model performance.
• Build practical tools for researchers and product teams: Develop clear user interfaces, documentation, evaluation reports, and diagnostic capabilities. Enable scientists and product teams to run evaluations, understand failures, interpret results, and extend tools without depending on the original developer.
• Own production reliability, performance, and responsible data use: Integrate automated testing, deployment, and monitoring from the beginning. Address performance, scalability, and operating costs, establish service maintenance and support ownership, and work with data specialists to implement access controls, permitted-use rules, and retention requirements.
• Collaborate, mentor, and guide engineering decisions: Partner with foundational research, data engineering, platform engineering, and product teams to align interfaces, responsibilities, and delivery plans. Resolve technical dependencies, mentor engineers, communicate trade-offs and delivery risks, and recommend changes to scope or technical approach when necessary.

Conocimientos

Software engineering experience

Python programming

Technical leadership

Collaboration with researchers

ML model testing & evaluation

Cloud engineering (AWS/Azure)

SQL/data processing

Communication & mentoring

Bachelor's degree in CS/Engineering

Educación

Bachelor's degree in Computer Science, Engineering, or related field

Herramientas

APIs

Python software

Docker

Descripción del empleo

Thomson Reuters Labs seeks a Lead Research Engineer to design and deliver production-ready software for AI model evaluation and benchmarking. You will build scalable evaluation infrastructure that enables research and product teams to compare large language models, measure quality, cost, and latency, and help select models.

You will own the engineering approach from prototypes to reliable tools, collaborating with researchers, engineers, and product partners while shaping technical direction and