Member of Technical Staff, Product Engineer/Full Stack

Judgment Labs Inc.; San Francisco (CA); On-site; USD 140,000 - 230,000; Full time

Job summary

Judgment Labs in San Francisco is seeking a seasoned engineer to own problems end-to-end: talking to customers, defining what to build, building it, and iterating until it’s great. You’ll design investigation interfaces, verify system changes with hosted environments, and build workflows that turn production data into datasets, evaluations, and regression checks.

You’ll help shape workspaces, roles, permissions, billing, and usage across multiple environments, enabling teams to operate

Qualifications

• 5+ years of experience building and scaling end-to-end production systems.
• Able to work across product, data, backend, infra and customer environments.
• Excellent communication across technical and non-technical audiences.
• Comfortable working directly with customers.

Responsibilities

• Design investigation interfaces to help engineers understand what systems did and why.
• Build a verification platform with hosted simulated environments and monitors for unintended changes.
• Create workflows turning production data into datasets, evaluations, and regression checks.
• Develop workspaces, roles, permissions, billing, usage, and limits for teams across multiple environments.

Skills

End-to-end systems

Customer engagement

Problem-solving

Communication

Job description

Judgment Labs is building the infrastructure for continual learning in long-horizon AI agents.

We turn long agent trajectories into clean, structured data for evals, labeling, rubric generation, context engineering, and RL workflows. Instead of only showing teams what happened, Judgment helps decide what matters, what should be learned from, and how that learning should flow back into the agent.

We’ve raised $32M across two rounds led by Lightspeed.

The Role

You’ll own problems end-to-end: talking to customers, defining what to build, building it, and iterating until it’s great. This is not a role where you implement specs handed down.

What You’ll Do

Investigation interfaces: Design how engineers understand what their systems did and why. Long traces, tool calls, decisions, failures. How do you make a complex sequence of events legible in minutes?

Verification: Build the platform for verifying system changes: hosted simulated environments, trajectory replay, and monitors for unintended behavior changes.

The improvement loop: Build the workflows that turn production data into datasets, evaluations, and regression checks, so the path from "found a problem" to "verified a fix" feels like one motion.

The platform underneath: Workspaces, roles, permissions, billing, usage, and limits for teams running many workflows across many environments.

What We’re Looking For

5+ years of experience building and scaling end-to-end production systems, from data layer to UI.

Strong technical problem-solving skills, especially in fast-changing, ambiguous environments.

A builder and tinkerer's mindset with high agency - you find creative ways to overcome obstacles and ship.

Comfort working directly with customers to understand their needs and solve real-world problems.

Excellent communication skills - clear, direct, and persuasive across technical and non-technical audiences.

Why Judgment?

You’ll work on problems customers actually feel: Engineers talk directly to teams building production agents, see where their systems fail, and turn those failures into product and infrastructure.

Small team, high ownership: You will not own a narrow slice. You’ll shape core systems early, ship quickly, and work across product, data, backend, infra, and customer environments.

In person in San Francisco: We work together in person because the problems are hard, the product is moving fast, and the feedback loops matter.