Staff ML Engineer, Agent Evaluation Platform

Moveworks; Mountain View (CA); Hybride; USD 180 000 - 240 000; Plein temps

Résumé du poste

Moveworks is seeking a Staff Software Engineer to own the agent evaluation platform. You will design the judgement layer for scoring multi-step agent actions across enterprise systems, calibrate against human labels, and create signals that drive improvements.

The role spans eval orchestration at scale, agent observability, and stateful simulation, with back-end work in Python/Go, OpenTelemetry, and contract-tested simulations in a high-scale enterprise context.

Qualifications

• 8+ years building production backend or infrastructure systems.
• Strong in Python or Go (ideally both).
• Experience designing and operating systems that handle real traffic at scale.
• Comfort turning non-deterministic agent behavior into a measurable signal.
• Experience in at least 3 of the listed areas (distributed systems, orchestration, observability, concurrency, data pipelines, gRPC/protobuf).

Responsabilités

• Eval orchestration at scale: run multi-turn agent scenarios end-to-end and manage environments.
• Scheduling, retries, high-concurrency execution, and run isolation at production data sizes.
• Versioned specs, datasets, and run-to-run comparisons as core operations.
• Consolidate evals onto a single orchestration service with a single source of truth.
• Establish reliability floors and an SLO for the harness.
• Enable self-serve evals without bespoke integrations.
• Lead OpenTelemetry-native observability for the agent platform.
• Define span data model for agent trajectories and cross-run diffing.
• Maintain trace context across async boundaries and sessions.
• Preserve prompts/completions through pipelines and provide debugging surfaces.
• Build stateful simulators with hermetic per-run setups.
• Create fakes for enterprise systems and contract-testing mocks.
• Lay the foundation for using eval signal to optimize the agent.

Connaissances

Distributed systems

Orchestration

Observability internals

Concurrent programming

Data pipelines

Python

Go

gRPC

Protobuf

OpenTelemetry

Outils

OpenTelemetry

gRPC

Protobuf

Description du poste

Moveworks is seeking a Staff Software Engineer to own the agent evaluation platform. You will design the judgement layer for scoring multi-step agent actions across enterprise systems, calibrate against human labels, and create signals that drive improvements.

The role spans eval orchestration at scale, agent observability, and stateful simulation, with back-end work in Python/Go, OpenTelemetry, and contract-tested simulations in a high-scale enterprise context.