Technical Lead - Autonomy Evaluation

Atoms; San Francisco (CA); On-site; USD 185,000 - 242,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

Disability insurance

Life insurance

401(k)

Equity

Sick time

Paid holidays

Team lunch

Job summary

Atoms is building real-world robots for food, mining, and transport, and seeks a Technical Lead for Autonomy Evaluation to own the evaluation metrics and log replay platform used for every candidate release. You will scale the pipeline from on-road testing to full regression across logs, enabling reliable score-based release decisions.

The role demands leadership in metric definitions, test design, and engineering mentorship, with onsite work in San Francisco and total rewards including equity

Qualifications

• Bachelor's degree in Computer Science, Robotics, Electrical Engineering, Statistics, or related field.
• 8+ years of relevant experience.
• 2+ years as a technical lead or Engineering manager for an evaluation, validation, or test team.
• Experience with offline or system-level evaluation for autonomous vehicles or robotics.
• Experience designing safety or behavior metrics for autonomous systems.
• Hands-on experience building log replay at scale.
• Experience with GPU clusters and large-scale compute.
• Experience with evaluation dataset design and benchmarking.
• Strong understanding of sensing, perception, localization, planning, and controls.
• Proficiency in Python and ability to read C++ code.

Responsibilities

• Own safety and behavior metrics a release is judged on, including collision and near-miss measures.
• Own the pipeline that converts logs into scored test cases for perception, localization, and planning.
• Develop regression and benchmarking across the log corpus for every candidate release.
• Oversee data mining of the corpus for rare, valuable events and sampling strategies.
• Ensure deterministic and reproducible execution of evaluation at scale.
• Set practices for metric definitions, test design, and result write-ups.
• Mentor engineers joining the function and participate in hiring.

Skills

Python

C++

Autonomous systems knowledge

Statistics for validation

Leadership

Education

Bachelor's degree in Computer Science, Robotics, Electrical Engineering, Statistics

Tools

GPU clusters

Robotics middleware

Logging and replay tooling

Job description

Atoms is building the machines that power the next era of progress.

Over the last decade, software has transformed the digital world. But the physical world, where food is made, minerals are mined, goods are moved, and industries are run, remains far less intelligent, far less efficient, and far more constrained. We’re changing that.

Atoms builds Physical AI— real-world robots for the industries that move civilization forward, starting with food, mining, and transport. Our systems are designed to understand, predict, and control the real world with precision, turning complex physical operations into something more reliable, more scalable, and more productive.

This work requires more than robotics. It requires deep integration across hardware, software, AI, operations, manufacturing, and real estate. We don’t just build machines in a lab. We deploy them into real environments, operate them, learn from them, and improve them until they work at scale.

We are roboticists, engineers, operators, and builders. We believe the next great technology companies will not only transform information, but the physical systems that shape everyday life.

If you want to work on hard problems with real-world impact, join us.

About the role

We are seeking a Technical Lead for Autonomy Evaluation to own the metrics our autonomy releases are judged on and the log replay and evaluation platform that computes them. In this role, you will take evaluation from recorded logs, to scored regression suites running on every candidate release, to the report a release passes before it reaches a vehicle, developing how replay, scoring, metrics, and test coverage come together into one platform engineers use every day. You will do this across on-road vehicles, and you will build the evaluation infrastructure that lets each new software release, sensor configuration, and platform be assessed seamlessly.

What you’ll do

• Metrics. Own the safety and behavior metrics a release is judged on, including collision and near-miss measures, trajectory agreement against ground truth, and comfort, and the go/no-go release criteria built on them.
• Log replay and scoring. Own the pipeline that converts recorded logs into scored test cases, covering open-loop and closed-loop evaluation of perception, localization, and planning against ground truth.
• Regression and benchmarking. Own the system that benchmarks new software against old across the log corpus for every candidate release, including evaluation dataset design, regression detection, and triage of a regressed case to an owner.
• Test coverage and data mining. Own how the corpus is mined for the long tail of rare and important events, including sampling strategy, selection of logs by expected value per replay-hour, and coverage of scenario classes the corpus does not yet contain.
• Execution at scale. Own deterministic and reproducible execution of evaluation, run identity, traceability of every result to a software version, and cost per replay-hour as a tracked number.
• Engineering practices. Set the practices for metric definitions, test design, code review, and the write-up of evaluation results.
• Technical standard. Mentor the engineers who join the function, set the technical bar for their work, and participate in hiring.

What we’re looking for

• Bachelor's degree in Computer Science, Robotics, Electrical Engineering, Statistics, or a related field with 8+ years of relevant experience.
• 2+ years as a technical lead or Engineering manager for an evaluation, validation, or test team.
• Ownership of an offline or system-level evaluation system for an autonomous vehicle or robotics program, or a comparable large-scale ML model evaluation system in production, with accountability for the metrics and for the release decisions made on them.
• Experience designing safety or behavior metrics for an autonomous system and defending them to engineering and executive audiences.
• Hands-on experience building log replay at scale, including scoring against ground truth in open-loop and closed-loop evaluation.
• Experience running evaluation pipelines on GPU clusters or comparable large-scale compute on a recurring cadence, with responsibility for reproducibility, throughput, and compute cost.
• Experience with evaluation dataset design, benchmarking, and regression detection across software or model versions.
• Working command of applied statistics for system validation, including statistical uncertainty, sampling, rare-event analysis, and the data volume a given claim requires.
• Working understanding of autonomous systems end to end, including sensing, perception, localization, planning, and controls.
• Experience on an autonomous vehicle or robotics platform.
• Working experience with a robotics middleware and its logging and replay tooling.
• Experience handling autonomous driving or robotics sensor data across multiple modalities and timestamps, including camera, LiDAR, and radar.
• Proficiency in Python and the ability to read, navigate, and debug existing C++ codebases.

At Atoms, you’ll work on one of the defining challenges of our time—bringing automation into the physical world to drive real, lasting impact. We exist to uncover valuable unknown truths and turn them into progress, which means constantly pushing beyond what’s known and building what doesn’t yet exist. The work is ambitious and often challenging, but it’s grounded in a shared sense of purpose and a team committed to seeing it through together. Our work only matters if it serves others, and we know that meaningful progress depends on the trust of the people we serve and the strength of our team—so we invest in both, creating an environment where you can do your best work and grow.

What else you need to know

This role is based in our San Francisco office location. As a company driven by innovation and continuous change, close collaboration is essential. We’re constantly reimagining our industry, creating new products, and refining our processes, and we do our best work together. That’s why all of our office-based teams work onsite, five days a week.

The base salary range for this role is $185,000 - $242,000 per year.

Actual compensation will be determined on an individual basis and may vary depending on experience, skills, and qualifications.

Base salary is just one part of your total rewards package. You may also be eligible for equity awards.

Benefits Summary (USA Full-Time Exempt Employees):

• Medical, Dental, Vision, Disability, and Life Insurance
• Flexible Spending Account / Health Savings Account Options
• 401(k)
• Equity
• Sick Time, Unlimited Flexible Time Off, and Paid Holidays
• Team lunch in our SoMa office every Tuesday and Thursday

Benefits are subject to change at the company's discretion.

#LI-Onsite

Voluntary Self-Identification

For government reporting purposes, we ask candidates to respond to the below self-identification survey.Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiringprocess or thereafter. Any information that you do provide will be recorded and maintained in aconfidential file.

As set forth in Atoms’s Equal Employment Opportunity policy,we do not discriminate on the basis of any protected group status under any applicable law.

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection.As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measurethe effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categoriesis as follows:

A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.