Staff Engineer, Autonomy Evaluation (VLA)

Cariadinc; Mountain View (CA); On-site; USD 191,267 - 269,203; Full time

Benefits offered by this job

401k with employer match

Tuition reimbursement

Unique vehicle lease program

Job summary

Cariadinc in Mountain View, California is seeking a Staff Engineer for Autonomy Evaluation to lead the development of evaluation methods and metrics for automated driving. This role is crucial for validating driving performance through structured evaluation and cross-functional collaboration.

The ideal candidate will have a Master's degree and extensive experience in metrics and evaluation frameworks. Benefits include competitive salary and comprehensive healthcare options.

Qualifications

• 8+ years of experience in autonomy, robotics, machine learning, or evaluation for complex technical systems.
• 3+ years of experience building metrics, validation frameworks, or evaluation pipelines for ML-based systems.

Responsibilities

• Define and evolve evaluation strategies for VLA / VLAM-based driving performance.
• Develop scalable evaluation pipelines across offline logs, simulation, and in-vehicle test results.
• Partner with ML and integration engineers to assess model quality and prioritize fixes.

Skills

Systems-level thinking

Strong analytical skills

Cross-functional communication

High ownership mindset

Education

Master’s degree in Computer Science, Robotics, Electrical Engineering, Applied Mathematics, or a related field

Tools

Python

Job description

Role Summary

The Staff Engineer, Autonomy Evaluation (VLA) leads the development of evaluation methods, metrics, and tooling for VLA / VLAM-based automated driving at CARIAD.

This role turns model behavior into clear performance evidence across route-level, scenario-level, and regression testing. You will build the evaluation loop that connects data, simulation, model iteration, and in-vehicle performance, helping the team improve driving quality quickly and make confident technical decisions.

You will work closely with ML, systems, and integration engineers to define how progress is measured for both demo performance and future productization.

Role Responsibilities

Evaluation Strategy & Metrics

• Define and evolve evaluation strategies for VLA / VLAM-based driving performance
• Design metrics and success criteria spanning safety, comfort, robustness, and generalization
• Create benchmarks for route-level, scenario-level, and long-tail/edge-case behavior

Evaluation Pipelines & Tooling

• Develop scalable evaluation pipelines across offline logs, simulation, and in-vehicle test results
• Build tooling for dataset slicing, scenario curation, and automated report generation
• Enable reproducible evaluation with clear versioning of data, models, and metrics

Regression Detection & Failure Analysis

• Detect and diagnose regressions across model iterations and software releases
• Establish a structured forensic / root-cause analysis workflow for driving failures and edge cases
• Translate findings into actionable improvements for model training, data collection, and system integration

Cross-Functional Partnership & Decision Support

• Partner with ML and integration engineers to assess model quality and prioritize fixes
• Support release and demo readiness decisions with clear, quantitative evidence
• Communicate results and trade-offs to technical and non-technical stakeholders

Technical Leadership & Standards

• Define best practices for evaluation methodology, metrics hygiene, and reporting
• Mentor engineers and raise the bar for rigor and speed of iteration in the evaluation loop
• Contribute to long-term evaluation architecture for productization

General Skills

• Systems-level thinking and ability to translate ambiguous driving behavior into measurable outcomes
• Strong analytical skills in benchmarking, regression analysis, and experiment design
• Ability to connect quantitative findings to practical engineering decisions and release readiness
• Strong cross-functional communication skills; comfortable working across teams and time zones
• High ownership mindset with rigor, curiosity, and a bias toward action

Required Specialized Skills

• Deep understanding of autonomy evaluation for ADAS / AD, robotics, or embodied AI systems
• Strong experience with performance metrics, benchmarking, regression analysis, and scenario-based validation
• Strong software engineering skills in Python; experience building robust evaluation or data pipelines
• Experience working with simulation, large-scale driving data, or model-in-the-loop evaluation

Desired Skills

• Experience with foundation models, VLMs / VLAMs, or end-to-end driving models
• Familiarity with learned evaluators, LLM/VLM-assisted evaluation, or agentic workflows for analysis and triage
• Experience in autonomous driving, robotics, or other safety-critical systems

Workplace Flexibility

• Collaborate across time zones; occasional early/late meetings to align with global partners
• Occasional travel as needed for vehicle testing, integration workshops, or demos

Years of Relevant Experience

• 8+ years of experience in autonomy, robotics, machine learning, or evaluation for complex technical systems
• 3+ years of experience building metrics, validation frameworks, or evaluation pipelines for ML-based systems

Required Education

• Master’s degree in Computer Science, Robotics, Electrical Engineering, Applied Mathematics, or a related field

Desired Education

• PhD in Computer Science, Robotics, Electrical Engineering, Applied Mathematics, or a related field

Compensation

Salary range is dependent on factors such as geographical differentials, credentials or certifications, industry-based experience, qualification and training. In the city of Mountain View, California, the salary range for this position is $191,267 - $269,203.

CARIAD, Inc. provides performance based merits and annual bonus along with a competitive benefits package. Benefits include medical, dental, vision, 401k with employer match and defined contribution plan, short and long term disability, basic life and AD&D insurance, employee assistance program, tuition reimbursement and student loan repayment plans, maternity and non-primary caregiver leave, adoption assistance, employee referral program and vacation and paid holidays. We also offer a unique vehicle lease program that covers registration and insurance fees.

Equal Opportunity Employer

CARIAD is an Equal Opportunity Employer. We welcome and encourage applicants from all backgrounds, and do not discriminate based on race, sex, age, disability, sexual orientation, nationalorigin, religion, color, gender identity/expression, marital status, veteran status, or any other characteristics protected by applicable laws.

Compliance

Employment with CARIAD Inc. is subject to export control and sanctions compliance. Some positions may involve access to technology and/or software source code subject to U.S. legal restrictions on release to certain foreign persons based on citizenship or permanent residence. To ensure compliance, applicants will be required to provide information for screening. Employment may be contingent on the outcome, including verification of U.S. citizenship or lawful permanent resident status, or confirmation that a license, exemption, or exception applies. CARIAD retains the discretion to decline to obtain a required license in any case. By applying, you acknowledge and agree to participate in this process.