AV Evaluation Manager

Jobtailor; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Jobtailor, based in Sunnyvale, leads a team focused on autonomy evaluation platforms and analysis tooling. We seek a leader to set technical direction across perception, prediction, and planning evaluation for autonomous driving systems, driving ML-based methodologies and robust dashboards for leadership insight.

The role emphasizes building a high-performing team, shaping roadmaps, and partnering with cross-functional teams to ensure delivery of scalable evaluation pipelines and risk-aware

Qualifications

• 8+ years of relevant experience in robotics, autonomous systems software, data analysis, ML evaluation, or autonomy analytics.
• Substantial experience leading technical teams and delivering complex software systems.
• 3+ years evaluating dynamic systems using numerical and/or ML approaches.
• Experience with time-series data, state derivatives, dynamics, and interconnected subsystems.
• Strong proficiency developing Python in production team environments, including testing, performance, and code review.
• Strong hands-on familiarity with Pandas, NumPy, SciPy, and data visualization libraries.
• Comfort working with C++ codebases, including reading, debugging, and instrumenting core algorithms.
• Demonstrated technical leadership driving architectural decisions, influencing cross-team designs, and owning complex services or platforms end-to-end.
• Strong cross-functional communication and ability to convert ambiguous evaluation needs into clear technical plans.
• Strong analytical curiosity and disciplined root-cause analysis of anomalous data and system discrepancies.
• Bachelor’s, Master’s, or PhD in Computer Science, Robotics, Mechanical or Aerospace Engineering, Machine Learning, Data Science, or a related field, or equivalent practical experience.
• Preferred: experience in autonomous driving or field robotics.
• Preferred: experience evaluating robotics or AV systems using camera, lidar, and radar sensor data.
• Preferred: statistical modeling, experimental design, and hypothesis testing.
• Preferred: proficiency in SQL and experience with logging, data schemas, and evaluation pipelines.
• Preferred: experience with ROS or similar robotics/IPC frameworks.
• Preferred: computational geometry, linear algebra, PyTorch, and ML techniques applied to perception, prediction, planning, or control.
• Preferred: experience with release gating, risk assessment, and safety-related autonomy decisions.
• Preferred: AI-assisted development and analytics tools.

Responsibilities

• Lead, coach, and grow a team of engineers building autonomy evaluation platforms, metrics, workflows, dashboards, and analysis tooling.
• Set technical direction for systems that introspect autonomous driving software performance across interfaces and the autonomy stack.
• Drive design and delivery of analysis algorithms that summarize, aggregate, and cluster metrics from simulation and on-road runs.
• Guide development of statistical and machine learning methods to quantify performance and identify behavior patterns across scenes and operational domains.
• Oversee evaluation approaches for ML components across perception, prediction, and planning.
• Ensure delivery of dashboards and interactive reports for trend analysis, drift detection, scenario coverage, and leadership insight.
• Partner with Autonomy, Simulation, Systems, and Safety teams to define requirements, resolve handoff issues, and align evaluation strategy with product and release needs.
• Help leverage VLMs and LLMs to classify autonomy performance and prioritize validation work with human-in-the-loop review.
• Maintain high standards through system design, code review, testing, observability, and engineering best practices.
• Translate system-level results into clear feedback for engineering and leadership.
• Build a high-performing, inclusive engineering team through hiring, coaching, feedback, and career development.
• Create clarity across priorities, roadmaps, and dependencies.
• Balance near-term delivery with long-term platform investments.
• Raise execution quality through clear ownership, design reviews, and operating mechanisms.
• Represent the team across partner organizations and influence system-level AV quality decisions.

Skills

Technical Leadership

Machine Learning Evaluation

Python Development

Data Analysis

Autonomous Systems Expertise

Cross-Functional Communication

Root-Cause Analysis

Education

Bachelor’s/Master’s/PhD in CS/Robotics/ML/DS

Tools

ROS

PyTorch

C++

SQL

Data Visualization Libraries

Pandas

NumPy

SciPy

Logging Tools

Job description

Lead, coach, and grow a team of engineers building autonomy evaluation platforms, metrics, workflows, dashboards, and analysis tooling
Set technical direction for systems that introspect autonomous driving software performance across interfaces and the autonomy stack
Drive design and delivery of analysis algorithms that summarize, aggregate, and cluster metrics from simulation and on-road runs
Guide development of statistical and machine learning methods to quantify performance and identify behavior patterns across scenes and operational domains
Oversee evaluation approaches for ML components across perception, prediction, and planning
Ensure delivery of dashboards and interactive reports for trend analysis, drift detection, scenario coverage, and leadership insight
Partner with Autonomy, Simulation, Systems, and Safety teams to define requirements, resolve handoff issues, and align evaluation strategy with product and release needs
Help leverage VLMs and LLMs to classify autonomy performance and prioritize validation work with human-in-the-loop review
Maintain high standards through system design, code review, testing, observability, and engineering best practices
Translate system-level results into clear feedback for engineering and leadership
Build a high-performing, inclusive engineering team through hiring, coaching, feedback, and career development
Create clarity across priorities, roadmaps, and dependencies
Balance near-term delivery with long-term platform investments
Raise execution quality through clear ownership, design reviews, and operating mechanisms
Represent the team across partner organizations and influence system-level AV quality decisions

Requirements

• 8+ years of relevant experience in robotics, autonomous systems software, data analysis, ML evaluation, or autonomy analytics
• Substantial experience leading technical teams and delivering complex software systems
• 3+ years evaluating dynamic systems using numerical and/or ML approaches
• Experience with time-series data, state derivatives, dynamics, and interconnected subsystems
• Strong proficiency developing Python in production team environments, including testing, performance, and code review
• Strong hands-on familiarity with Pandas, NumPy, SciPy, and data visualization libraries
• Comfort working with C++ codebases, including reading, debugging, and instrumenting core algorithms
• Demonstrated technical leadership driving architectural decisions, influencing cross-team designs, and owning complex services or platforms end-to-end
• Strong cross-functional communication and ability to convert ambiguous evaluation needs into clear technical plans
• Strong analytical curiosity and disciplined root-cause analysis of anomalous data and system discrepancies
• Bachelor’s, Master’s, or PhD in Computer Science, Robotics, Mechanical or Aerospace Engineering, Machine Learning, Data Science, or a related field, or equivalent practical experience
• Preferred: experience in autonomous driving or field robotics
• Preferred: experience evaluating robotics or AV systems using camera, lidar, and radar sensor data
• Preferred: statistical modeling, experimental design, and hypothesis testing
• Preferred: proficiency in SQL and experience with logging, data schemas, and evaluation pipelines
• Preferred: experience with ROS or similar robotics/IPC frameworks
• Preferred: computational geometry, linear algebra, PyTorch, and ML techniques applied to perception, prediction, planning, or control
• Preferred: experience with release gating, risk assessment, and safety-related autonomy decisions
• Preferred: AI-assisted development and analytics tools

Core Competencies

Demonstrates expertise in leading technical teams to develop and evaluate autonomous systems, utilizing machine learning and data analysis to drive performance insights. Proficient in Python, statistical modeling, and cross-functional communication to align technical strategies with product goals.

Highest-signal resume keywords

• Technical Leadership
• Machine Learning Evaluation
• Python Development
• Data Analysis
• Autonomous Systems Expertise

Hard Skills

• Machine Learning
• Data Analysis
• Statistical Modeling
• Python
• C++
• SQL
• Time-Series Data
• Experimental Design
• Computational Geometry
• Linear Algebra

Soft Skills

• Cross-Functional Communication
• Analytical Curiosity
• Coaching
• Feedback
• Team Building

Industry Keywords

• Autonomous Driving
• Robotics
• AV Systems
• Sensor Data
• Risk Assessment

Tools & Technologies

• Pandas
• NumPy
• SciPy
• Data Visualization Libraries
• ROS
• PyTorch
• Logging Tools
• Evaluation Pipelines
• Dashboards
• Interactive Reports