Staff Automation Engineer for Autonomy & ML Validation

Shield; San Diego (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

Shield AI in San Diego is seeking a Staff Automated Test Engineer to provide technical leadership for automated verification across our Hivemind autonomy platform. You will define test architecture, validation infrastructure, CI/CD pipelines, and ML quality systems that enable resilient unmanned aircraft and robotic capabilities.

You will collaborate across software, machine learning, simulation, and hardware teams to build scalable test frameworks, ML validation pipelines, and observability

Qualifications

• 8+ years in software engineering, test infra, or related fields.
• 5+ years building scalable automation frameworks in Python.
• Proven track record designing cross-team test/CI/CD/MLOps infra.
• Experience validating ML systems across data, training, deployment, monitoring.
• Understanding of ML quality risks: data leakage, drift, distribution shift.
• Experience with GPU-accelerated infra and Kubernetes workloads.
• Strong Python for asynchronous/concurrent programming.
• Familiar with Linux development environments.

Responsibilities

• Own the technical strategy, architecture, and roadmap for automated testing, verification, and MLOps quality across the Hivemind ecosystem.
• Design and maintain scalable test frameworks for autonomy software, backend services, APIs, and distributed hardware environments.
• Lead functional, integration, regression, system, performance, reliability, and end-to-end testing across simulation, edge-compute, software-in-the-loop, and hardware-in-the-loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness, regression, latency, resource utilization, and system integration.
• Establish CI/CD and continuous training workflows that provide versioning and traceability for datasets, models, configurations, evaluation results, and deployment artifacts.
• Develop scenario-based validation for autonomy models, including edge cases, degraded sensing or communications, distribution shifts, and representative mission conditions.
• Create observability, analytics, and failure-triage capabilities for software behavior, model and data drift, inference health, test results, and production performance.
• Build Python automation that improves test execution, parallelization, reporting, environment setup, experiment comparison, and developer productivity.
• Create test harnesses, simulators, stubs, mocks, and synthetic data capabilities that improve system testability and coverage.
• Collaborate with software, autonomy, machine learning, data, simulation, and systems engineers to define verification strategies and improve designs before implementation.
• Develop and govern AI-assisted engineering workflows using coding agents and LLM-based tools for test generation, log analysis, debugging, and failure triage while maintaining security, reproducibility, and traceability.

Skills

Python automation

MLOps

Distributed testing

CI/CD pipelines

GPU workloads

Linux

Test architecture

Observability

LLM tooling

Automation frameworks

Tools

Conan

Kubernetes

Kubeflow

CUDA

Nix

PyTest

Job description

Shield AI in San Diego is seeking a Staff Automated Test Engineer to provide technical leadership for automated verification across our Hivemind autonomy platform. You will define test architecture, validation infrastructure, CI/CD pipelines, and ML quality systems that enable resilient unmanned aircraft and robotic capabilities.

You will collaborate across software, machine learning, simulation, and hardware teams to build scalable test frameworks, ML validation pipelines, and observability