Senior AI Validation Engineering Manager

Sonatus, Inc.; Sunnyvale (CA); Hybrid; USD 220,000 - 265,000; Full time

Benefits offered by this job

Hybrid work arrangement

Lunches, snacks & beverages on-site

Unlimited PTO

Life Insurance

Dependent Care Program

Job summary

Sonatus, Inc. seeks an experienced Senior Engineering Manager to build and lead the AI Validation function, spanning embedded, cloud, and LLM-based products. You will design scalable test strategies, governance mechanisms, and evaluation frameworks to enable trustworthy AI features at automotive scale.

You will lead a high-performing AI Validation team, collaborating with engineering, product, and safety stakeholders to ensure production readiness and quality in AI-enabled releases.

Qualifications

• Bachelor's degree in CS/CE/EE; MS preferred.
• 10+ years in software or systems engineering across embedded, cloud, networking, security, and automotive domains; with 3+ years leading engineering or QA teams.
• Hands-on experience with AI/ML systems, ML workflows, neural networks, and MLOps.
• Deep understanding of LLMs, RAG architectures, embeddings, retrieval optimization, and agentic AI frameworks.
• Proven experience designing scalable evaluation frameworks for AI systems, including multi-step agentic workflows and regression testing.
• Experience with hybrid evaluation methods combining deterministic validation and LLM-as-a-Judge techniques.
• Experience with RAG evaluation frameworks such as RAGAS.
• Experience validating hallucination, grounding, citations, bias, safety in production LLM apps.
• Experience testing cloud-native platforms and safety-critical systems.
• Experience establishing AI governance and Responsible AI practices.

Responsibilities

• Define and drive AI validation strategy across embedded, in-vehicle, cloud-connected and cloud-native AI systems.
• Lead, hire, mentor, and grow a high-performing AI Validation organization, establishing scalable processes and direction.
• Own end-to-end validation strategy for AI/ML models, LLMs, RAG pipelines, and agentic AI workflows.
• Architect scalable evaluation frameworks and benchmarking platforms for AI systems with multi-step agentic workflows.
• Design evaluation harnesses for RAG and agentic systems measuring grounding, citations, and safety.
• Evaluate and integrate open-source and commercial AI validation technologies; guide build-versus-buy decisions.
• Establish AI governance, safety, compliance, and Responsible AI for automotive/enterprise AI products.
• Act as quality gatekeeper for AI releases; coordinate with engineering, product, safety and OEM stakeholders.
• Collaborate to define validation strategies for new AI capabilities and standardize practices.
• Stay current with AI evaluation advances to scale validation capabilities across Sonatus.

Skills

Python

Linux

Test frameworks

JIRA

AI validation

Education

Bachelor's in CS/EE/CE

MS preferred

Tools

Jenkins

Playwright

PyTest

Behave

LangGraph

Job description

Sonatus, Inc. seeks an experienced Senior Engineering Manager to build and lead the AI Validation function, spanning embedded, cloud, and LLM-based products. You will design scalable test strategies, governance mechanisms, and evaluation frameworks to enable trustworthy AI features at automotive scale.

You will lead a high-performing AI Validation team, collaborating with engineering, product, and safety stakeholders to ensure production readiness and quality in AI-enabled releases.