Autonomy Evaluation Tech Lead

Atoms; San Francisco (CA); On-site; USD 185,000 - 242,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

Disability Insurance

Life Insurance

401(k) Plan

Equity

Paid Holidays

Parental Leave

Commuter Benefit

Team lunch

Job summary

Atoms is seeking a Technical Lead for Autonomy Evaluation to own metric definitions and the log replay/evaluation platform for autonomous systems. You will guide evaluation from logs to scored test cases and ensure release decisions are well-supported by data.

You will lead onset of regression benchmarking, data mining for rare events, and scalable execution across GPU clusters, influencing the team and processes.

Qualifications

• Bachelor's degree in Computer Science, Robotics, Electrical Engineering, Statistics, or related field.
• At least 8+ years of relevant experience.
• Experience as a technical lead or engineering manager for evaluation/testing.
• Ownership of offline/system-level evaluation for autonomous systems or large-scale ML models.
• Experience designing safety/behavior metrics for autonomous systems.
• Hands-on experience building log replay at scale and scoring against ground truth.

Responsibilities

• Own safety and behavior metrics used to judge releases.
• Own the log replay and scoring pipeline for evaluation.
• Benchmark software against previous releases and manage regression detection.
• Design and execute evaluation data mining across log corpus.
• Ensure deterministic, reproducible evaluation and track cost per replay-hour.
• Set engineering practices for metrics, test design, code review, and reporting.

Skills

Leadership

Metrics design

Log replay

Benchmarking

Statistics

Autonomous systems

Python

C++ reading

Debugging

Communication

Education

Bachelor's degree in CS/Robotics/EE/Statistics

8+ years experience

Lead/evaluation experience

Tools

Python

C++

Job description

Atoms is seeking a Technical Lead for Autonomy Evaluation to own metric definitions and the log replay/evaluation platform for autonomous systems. You will guide evaluation from logs to scored test cases and ensure release decisions are well-supported by data.

You will lead onset of regression benchmarking, data mining for rare events, and scalable execution across GPU clusters, influencing the team and processes.