Senior AI Evaluation Systems Engineer

Servicenow; Santa Clara (CA); On-site; USD 180,000 - 320,000; Full time

Job summary

Moveworks is seeking an experienced backend/infrastructure engineer to design the judgment layer for our agent evaluation platform. You will define rubrics, calibrate labels, and shape the signal used to improve agents across multi-step trajectories.

This role focuses on applied ML at scale with stateful enterprise simulations, robust observability, and reliable backends. Strong Python or Go skills and comfort with ambiguity are essential.

Qualifications

• 8+ years building production backend or infrastructure systems.
• Strong in Python or Go (ideally both).
• Experience designing and operating systems that handle real traffic at scale.
• Comfort making a non-deterministic system measurable; ML background not required but helpful.
• Comfort with ambiguity and evolving requirements.

Responsibilities

• Own the evaluation platform's rubrics, judges, and calibration against human labels.
• Build scalable orchestration for multi-turn agent scenarios and validation workflows.
• Lead observability efforts, tracing, and data collection for agent trajectories.
• Develop stateful simulation environments to reproduce enterprise systems and data.
• Contribute to reliability, reproducibility, and run-to-run comparisons across experiments.

Skills

Distributed systems

Orchestration/workflows

Observability internals

Async programming

Data pipelines

gRPC/protobuf design

Tools

OpenTelemetry

Temporal/Airflow

Job description

Moveworks is seeking an experienced backend/infrastructure engineer to design the judgment layer for our agent evaluation platform. You will define rubrics, calibrate labels, and shape the signal used to improve agents across multi-step trajectories.

This role focuses on applied ML at scale with stateful enterprise simulations, robust observability, and reliable backends. Strong Python or Go skills and comfort with ambiguity are essential.