Product Engineer: AI Agent Platform & Debugger

judgmentlabs; San Francisco (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Judgment is building the infrastructure for AI agents, delivering end-to-end production systems from data layers to UI. You will own the Judgment Agent workflow, verification environments, and UX to analyze large-scale production traces, with strong emphasis on customer-driven development and clear cross-functional communication.

You’ll design scalable interfaces and tooling that make complex agent behavior legible, enabling rapid iteration and robust deployment across teams and environments.

Qualifications

• Experience building and scaling end-to-end production systems from data layer to UI.
• Strong problem-solving skills in fast-changing, ambiguous environments.
• Builder mindset with high agency; ships solutions despite obstacles.
• Hands-on experience with LLMs or agents, or the drive to get there quickly.
• Comfort working directly with customers to understand needs and solve real-world problems.
• Excellent communication skills across technical and non-technical audiences.

Responsibilities

• Shape how the Judgment Agent runs large-scale investigations across thousands of production traces.
• Build the platform for verifying agent changes, including simulated evals and trajectory replay.
• Design interfaces to help engineers understand what their agents did and why, including long traces and decisions.
• Create swarm UX so engineers can monitor parallel investigations and steer findings.
• Develop workflows turning production trajectories into datasets, rubrics, and regression checks.
• Develop the platform underneath with workspaces, roles, billing, usage, and limits for multi-agent environments.
• Provide an SDK and terminal-first workflows enabling Judgment as a subagent during development.

Skills

End-to-end production systems

LLMs/agents experience

Customer collaboration

Problem solving in ambiguity

Builder mindset

Cross-functional communication

Job description

Judgment is building the infrastructure for AI agents, delivering end-to-end production systems from data layers to UI. You will own the Judgment Agent workflow, verification environments, and UX to analyze large-scale production traces, with strong emphasis on customer-driven development and clear cross-functional communication.

You’ll design scalable interfaces and tooling that make complex agent behavior legible, enabling rapid iteration and robust deployment across teams and environments.