Senior AI Quality Engineer for Cortex Code

Snowflake; Menlo Park (CA); On-site; USD 200,000 - 270,000; Full time

Job summary

Snowflake is hiring for a role focused on measuring the quality and efficiency of Cortex Code end to end, turning fuzzy signals into hard metrics and running experiments that move the needle for real coding and data-engineering workflows. You will work on a small, high-powered team impacting developers across Snowflake.

You will own evaluation harnesses, diagnose failure modes from real agent trajectories, and drive cost and latency improvements while maintaining quality at scale.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Statistics, or a related field.
• 6+ years of experience shipping software in production, including AI/LLM features.
• Fluency in at least one of Python, TypeScript, or Go, and willingness to work across all three.
• A measurement-first, systems-thinking instinct: you optimize for user outcomes over isolated metrics, and you can design an eval that is not fooling you (sampling, ground-truth quality, leakage, noise).
• Comfort debugging complex, unpredictable, real-world failures.
• Strong communication skills: you can make a quality or cost result legible to engineers, product, and leadership, and collaborate effectively in a team environment.

Responsibilities

• Take agents from prototype to production: design and refine agent behaviors for real coding and data-engineering workflows, and make them reliable enough to depend on.
• Own agent quality end to end: build eval harnesses, diagnose failure modes from real agent trajectories, and run experiments that hillclimb the metrics that matter.
• Drive down cost and latency without regressing quality: prompt caching, context compaction, tool-result offloading, and cheaper model routing for sub-tasks.
• Debug production failures and systematically increase robustness: close the loop from a customer’s broken run back to a fix and a regression test.
• Build the data pipelines that feed real-world task insights back into evals and modeling.
• Onboard and bake off frontier models: measure their strengths and failure modes, and decide where each belongs in the product.
• Partner with product and infra: shape user-facing agent behavior, and set the metrics and standards that define a successfully completed complex task.

Skills

Python

TypeScript

Go

AI/LLM features

Education

Bachelor's degree in CS/Engineering/Statistics

Job description

Snowflake is hiring for a role focused on measuring the quality and efficiency of Cortex Code end to end, turning fuzzy signals into hard metrics and running experiments that move the needle for real coding and data-engineering workflows. You will work on a small, high-powered team impacting developers across Snowflake.

You will own evaluation harnesses, diagnose failure modes from real agent trajectories, and drive cost and latency improvements while maintaining quality at scale.