AI Evaluation Architect for Production Agents

PhysicsX

Greater London

Presencial

GBP 90.000 - 120.000

Jornada completa

Hace 2 días
Sé de los primeros/as/es en solicitar esta vacante

Consigue una respuesta de este empleador

Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.

Supera los filtros ATS

Descripción de la vacante

PhysicsX is seeking a hands-on engineering leader in the UK to build the evaluation discipline for trustworthy agents in production. You will report to the engineering lead for our Agentic strategy and provide shared infrastructure, frameworks and coaching to enable teams to design, deploy and improve evaluations.

You will own cross-product evaluation, develop public benchmarks and ensure evaluation integrity across product, research and delivery.

Responsabilidades

  • Lead the technical strategy for how PhysicsX evaluates agents.
  • Build reusable evaluation frameworks, reference implementations and golden paths on top of existing tooling.
  • Help product, delivery and customer teams establish robust eval suites, then turn what works into repeatable guidance.
  • Develop datasets and evaluators combining deterministic checks, domain-expert judgement, model-based grading and human review.
  • Connect offline evaluation, regression testing and production telemetry for drift detection and learning from real use.
  • Define cross-product scorecards and shared benchmarks without taking ownership away from teams shipping each agent.
  • Create and publish credible open benchmarks for agents performing real engineering and manufacturing workflows.
  • Collaborate with Research and Post-Training on model and agent evaluation; ensure benchmark integrity across products and externally disclosed results.
  • Foster institutional discipline through reviews, examples, training and cross-team collaboration to embed rigorous evaluation as a core practice.

Descripción del empleo

PhysicsX is seeking a hands-on engineering leader in the UK to build the evaluation discipline for trustworthy agents in production. You will report to the engineering lead for our Agentic strategy and provide shared infrastructure, frameworks and coaching to enable teams to design, deploy and improve evaluations.

You will own cross-product evaluation, develop public benchmarks and ensure evaluation integrity across product, research and delivery.

Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

AI Evaluation Engineer: Agentic Benchmarks & Production
AI Evaluation Engineer: Agentic Benchmarks & Production

Teli Labs • Greater London

Presencial
GBP 90.000 - 130.000
AI Engineer: Evals
AI Engineer: Evals

PhysicsX • Greater London

Presencial
GBP 90.000 - 120.000
AI Engineer: Evals
AI Engineer: Evals

Teli Labs • Greater London

Presencial
GBP 90.000 - 130.000
Staff Research Engineer: Agent Evals & Post-Training (London)
Staff Research Engineer: Agent Evals & Post-Training (London)

Nominal • Greater London

Presencial
GBP 180.000 - 220.000
Unlimited PTO
Free meals
Dev stipend
+4
Head of AI Agent Engineering – Hands-On Lead
Head of AI Agent Engineering – Hands-On Lead

Glaxosmithkline • Gran Bretaña

Presencial
GBP 140.000 - 210.000
AI Architect
AI Architect

AutoThink Group • Rochdale

Híbrido
GBP 120.000 - 180.000
Senior Applied AI Engineer — Production AI & Client Impact
Senior Applied AI Engineer — Production AI & Client Impact

Scaleai • Greater London

Presencial
GBP 120.000 - 180.000
Senior Applied AI Engineer: Production, Evaluation & Impact
Senior Applied AI Engineer: Production, Evaluation & Impact

Scale AI • City Of London

Presencial
GBP 90.000 - 150.000
Senior AI Engineer – Agent Platform & Production Agents
Senior AI Engineer – Agent Platform & Production Agents

TaskEasy • Gran Bretaña

A distancia
GBP 90.000 - 140.000
Pension with employer match
Generous holiday allowance
Enhanced parental leave
+2
Agentic Evaluation Engineer — Benchmarks & Systems
Agentic Evaluation Engineer — Benchmarks & Systems

AI Startups UK • Greater London

Híbrido
GBP 90.000 - 130.000
Competitive salary
Equity ownership
Private healthcare
+1