Full-Stack Engineer - AI Agent Platform & Debug UX

Judgment Labs; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Judgment is building the learning infrastructure for AI agents. The role involves owning problems end-to-end—from talking to customers and defining what to build to implementing and iterating until the product is great—in a production environment.

You will shape large-scale investigations, verify agent changes, and design interfaces that make complex agent behavior legible to engineers. Expect a hands-on, customer-facing, high-agency role with broad impact.

Qualifications

• Experience building and scaling end-to-end production systems.
• Strong technical problem-solving skills in fast-changing environments.
• Builder mindset with high agency and hands-on problem solving.
• Hands-on experience with LLMs or agents or the drive to gain it fast.

Responsibilities

• Shape how Judgment Agent runs large-scale investigations across production traces.
• Build the platform for verifying agent changes with simulated environments and trajectory replay.
• Design interfaces to help engineers understand agent behavior and decisions.
• Create a swarm UX to monitor parallel investigations and guide findings.

Skills

End-to-end systems

Technical problem solving

Builder mindset

LLMs/agents experience

Customer collaboration

Clear communication

Tools

LLM platforms

Job description

Judgment is building the learning infrastructure for AI agents. The role involves owning problems end-to-end—from talking to customers and defining what to build to implementing and iterating until the product is great—in a production environment.

You will shape large-scale investigations, verify agent changes, and design interfaces that make complex agent behavior legible to engineers. Expect a hands-on, customer-facing, high-agency role with broad impact.