Senior Platform Engineer, Agent Oversight & Observability

Scale; San Francisco (CA); On-site; USD 216,000 - 270,000; Full time

Benefits offered by this job

Health, dental, vision

Retirement plan

Learning stipend

Generous PTO

Commuter stipend

Job summary

Scale is seeking a Software Engineer for Agent Oversight to build the production-ready infrastructure that lets our agents be observed, evaluated, and improved at scale. You will design core platform capabilities for deploying, monitoring, and evaluating agentic applications in production, and build APIs and data pipelines to capture telemetry and performance signals.

You will work with ML engineers to reason about model behavior, evaluation quality, and improvement loops, while owning the

Qualifications

• 4+ years of professional software engineering experience with backend/distributed systems, APIs and data pipelines
• Hands-on experience building production software for ML/LLM-powered products or platforms
• Knowledge of how ML systems behave in production, including evaluation signals and failure modes
• Experience partnering with ML engineers to turn prototypes into reliable platform capabilities
• Experience building infrastructure or platforms that other teams build on top of
• Track record of owning features end-to-end within a larger system
• Ability to work in a fast-changing domain and communicate effectively with cross-functional teams
• Willingness to give and receive feedback to help others grow

Responsibilities

• Design and build core platform capabilities for deploying, monitoring, and evaluating agentic applications in production
• Build reliable APIs and data pipelines that capture agent telemetry and performance signals at scale
• Collaborate with ML engineers to reason about model behavior, evaluation quality, and improvement loops
• Own the reliability, scalability, and observability of platform components serving multiple enterprise and government customers
• Work cross-functionally with product, forward deployed engineering, and customers to translate deployment requirements into platform features
• Develop features end-to-end: system design, implementation, debugging, and testing
• Participate in high-velocity experimentation to validate platform capabilities against real usage

Skills

Backend systems

APIs

Data pipelines

Observability

ML systems

Experimentation

Cross-functional collaboration

Problem solving

Independent work

Job description

Scale is seeking a Software Engineer for Agent Oversight to build the production-ready infrastructure that lets our agents be observed, evaluated, and improved at scale. You will design core platform capabilities for deploying, monitoring, and evaluating agentic applications in production, and build APIs and data pipelines to capture telemetry and performance signals.

You will work with ML engineers to reason about model behavior, evaluation quality, and improvement loops, while owning the