Senior Software Engineer — AI Production & Telemetry

Scale AI, Inc.; New York (NY); On-site; USD 216,000 - 270,000; Full time

Benefits offered by this job

Health, dental, vision coverage

Equity-based compensation

PTO and retirement benefits

Learning stipend

Commuter stipend

Job summary

Scale AI, Inc. is seeking a Software Engineer on Agent Oversight to design and build the platform infrastructure that allows production agents to be observed, evaluated, and improved at scale.

You will own end-to-end systems—from design to deployment—ensuring reliability and security in mission-critical workflows. You will collaborate with ML engineers and enterprise customers to translate deployment requirements into robust platform features, implement observability tooling, and contribute to

Qualifications

• 4+ years of professional software engineering experience in backend or distributed systems.
• Experience with production software for ML/LLM-powered platforms, including evaluation and observability.
• Familiarity with production ML workflows, evaluation signals, and data quality considerations.

Responsibilities

• Design and build core platform capabilities for deploying, monitoring, and evaluating agentic applications in production.
• Build reliable APIs and data pipelines for agent telemetry and evaluation signals at scale.
• Collaborate with ML engineers to turn prototypes into reliable platform features.
• Own the reliability, scalability, and observability of platform components serving enterprise and government customers.
• Translate real-world deployment requirements into platform features with cross-functional teams.
• Develop end-to-end: design, implement, test, and iterate on features.
• Participate in high-velocity experiments to validate platform capabilities.

Skills

Backend systems

APIs

Data pipelines

ML/LLM observability

Problem solving

Independent work

Tools

Observability tooling

Telemetry pipelines

Evaluation harnesses

Job description

Scale AI, Inc. is seeking a Software Engineer on Agent Oversight to design and build the platform infrastructure that allows production agents to be observed, evaluated, and improved at scale.

You will own end-to-end systems—from design to deployment—ensuring reliability and security in mission-critical workflows. You will collaborate with ML engineers and enterprise customers to translate deployment requirements into robust platform features, implement observability tooling, and contribute to