AI Automation Engineer: End-to-End Evaluations + Equity

Niantic Spatial, Inc.; San Francisco (CA); On-site; USD 158,000 - 210,000; Full time

Benefits offered by this job

Annual bonus

Equity

Medical coverage

Dental coverage

Vision coverage

401(k)

Job summary

Niantic Spatial, Inc. seeks an AI Automation Engineer in San Francisco to build and own the end-to-end evaluation system used by the Real-World Test Lab. You will replace manual processes with scalable automation, ensure results are trustworthy, and keep evaluations running across releases.

You will design reproducible pipelines, instrument scorecards, and automate agent-driven workflows, collaborating with data and backend teams to enable reliable testing of complex spatial AI systems.

Qualifications

• Built production-grade automation or test infrastructure used daily by engineers.
• Experience evaluating systems with non-binary correctness and measurable quality.
• Strong Python, orchestration, CI workflows, cloud storage, and reproducible environments.
• Able to document technical findings clearly for broad audiences.
• Bachelor's degree in a relevant field, or equivalent experience.

Responsibilities

• Own end-to-end automation for lab evaluations, including setup, orchestration, and data capture.
• Instrument scorecards to compare results across versions, devices, and conditions.
• Develop agent-driven workflows to scale realistic use cases.
• Convert ad-hoc experiments into standing, repeatable protocols.
• Provide actionable diagnostics with reproducible data attached.
• Collaborate with the AI Data Manager to ensure reproducible dataset states.

Skills

Python

Orchestration

CI/CD pipelines

Cloud storage

Reproducible environments

Systems evaluation

Automation engineering

Technical writing

Education

Bachelor's degree or equivalent

Tools

Backend services APIs

Job description

Niantic Spatial, Inc. seeks an AI Automation Engineer in San Francisco to build and own the end-to-end evaluation system used by the Real-World Test Lab. You will replace manual processes with scalable automation, ensure results are trustworthy, and keep evaluations running across releases.

You will design reproducible pipelines, instrument scorecards, and automate agent-driven workflows, collaborating with data and backend teams to enable reliable testing of complex spatial AI systems.