Staff Automation Engineer — AI/ML Testing & Validation

Lever, Inc.; San Diego (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

Shield AI is seeking a Staff Automated Test Engineer to lead verification for the Hivemind autonomy platform, spanning test architecture, validation infrastructure, ML ops quality, and CI/CD pipelines across software, simulation, and hardware-in-the-loop systems.

You will collaborate with software, ML, data, and systems teams to raise testing standards, implement scalable automation, and ensure traceability from data to deployment in GPS-denied environments.

Qualifications

• 8+ years of software engineering, test infrastructure, or systems verification experience.
• 5+ years building scalable automation frameworks or developer tooling in Python.
• Experience designing test, CI/CD, or MLOps infrastructure used across multiple teams.
• Experience validating machine learning systems across data, training, evaluation, and deployment lifecycles.
• Understanding of ML quality risks: data leakage, distribution shift, drift, and model regression.
• Experience with GPU-accelerated infrastructure, Kubernetes, and performance benchmarking.

Responsibilities

• Own the technical strategy, architecture, and roadmap for automated testing, verification, and MLOps quality.
• Design and maintain scalable test frameworks for autonomy software, backend services, APIs, and distributed environments.
• Lead end-to-end testing across simulation, edge-compute, software-in-the-loop, and hardware-in-the-loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, and model accuracy and robustness.
• Establish CI/CD and continuous training workflows with versioning for datasets, models, and configurations.
• Develop scenario-based validation for autonomy models including edge cases and distribution shifts.
• Collaborate with software, autonomy, ML, data, simulation, and systems engineers to define verification strategies.

Skills

Python automation

Distributed systems testing

Technical leadership

MLOps

CI/CD pipelines

Test architecture

Automation frameworks

Tools

Conan

pip

setuptools

Poetry

Nix

Kubernetes

Job description

Shield AI is seeking a Staff Automated Test Engineer to lead verification for the Hivemind autonomy platform, spanning test architecture, validation infrastructure, ML ops quality, and CI/CD pipelines across software, simulation, and hardware-in-the-loop systems.

You will collaborate with software, ML, data, and systems teams to raise testing standards, implement scalable automation, and ensure traceability from data to deployment in GPS-denied environments.