Senior Engineer, AI Agent Quality & Platform

ACM CAIS 2026; New York (NY); On-site; USD 190,000 - 270,000; Full time

Job summary

Databricks is building a new Staff Software Engineer – Agent Quality to lead the design and scaling of infrastructure, tooling, and workflows that rigorously evaluate AI agents across development, training, and production.

You will stand up foundational evaluation infrastructure for Genie Agents, enabling benchmarking, regression detection, and quality measurement across research and product teams, driving a cutting-edge open feedback loop for agent improvement.

Qualifications

• 6+ years industry experience building software systems.
• Strong Python programming skills, with experience building production or research infrastructure.
• Experience building or operating distributed systems, data pipelines, or large-scale infrastructure with emphasis on reliability and correctness.
• Ability to design rigorous systems that produce trustworthy signals for complex apps.
• Comfort across ambiguous research/product boundaries and partner with researchers and engineers to deliver robust internal platforms.
• High bar for technical quality, ownership, and influencing roadmap across teams.

Responsibilities

• Stand up the foundational evaluation infrastructure for Genie Agents, enabling benchmarking, regression detection, and quality measurement across research and product teams.
• Build the flywheel that connects evaluation results back into agent improvement—closing the loop between production signals, training, and development.
• Shape the long-term technical direction for agent quality infrastructure as Databricks expands its first‑party agents and development platform.
• Collaborate with researchers and engineers to turn ideas into robust internal platforms and tooling.
• Ensure reliability and scalability of evaluation pipelines and data flows across teams.

Skills

Python

Distributed systems

Research infrastructure

Ownership

Cross-team collaboration

Tools

CI/CD tooling

Observability tooling

Job description

Databricks is building a new Staff Software Engineer – Agent Quality to lead the design and scaling of infrastructure, tooling, and workflows that rigorously evaluate AI agents across development, training, and production.

You will stand up foundational evaluation infrastructure for Genie Agents, enabling benchmarking, regression detection, and quality measurement across research and product teams, driving a cutting-edge open feedback loop for agent improvement.