Production AI Engineer: Scalable Agent Systems

Judgment Labs; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 280,000; Full time

Job summary

Judgment Labs in San Francisco seeks Applied AI Engineers to build AI systems that analyze agent interaction data, measure behavior, and scale learning and feedback into production.

You will own end-to-end projects with real-world data, collaborating with product teams to deploy self-improving agent workflows and frontier AI research across finance, legal, and operations. Significant autonomy and impact on product direction are offered.

Qualifications

• Experience with real-world agent data and production systems.
• Familiarity with RL concepts and post-training workflows.
• Ability to design and optimize large-scale data pipelines for experiments.

Responsibilities

• Build AI systems to aggregate, index, and analyze large-scale long-running agent interaction data.
• Design post-training and optimization workflows to improve agents.
• Develop agent platform infrastructure including orchestration, runtimes, and developer tools.
• Create internal tools and infrastructure for rapid experimentation, analysis, and training.
• Collaborate with product to integrate agents into customer-facing workflows.
• Partner with external researchers on frontier AI research.

Skills

Data quality

Evaluation

Agent systems

Reinforcement learning

Infrastructure

Job description

Judgment Labs in San Francisco seeks Applied AI Engineers to build AI systems that analyze agent interaction data, measure behavior, and scale learning and feedback into production.

You will own end-to-end projects with real-world data, collaborating with product teams to deploy self-improving agent workflows and frontier AI research across finance, legal, and operations. Significant autonomy and impact on product direction are offered.