Staff Automation Engineer — ML Testing & CI/CD

Speedrun Talent Network; San Mateo (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Shield AI is seeking a Staff Automated Test Engineer to lead automated verification for its Hivemind autonomy platform. You will shape test architecture, MLOps quality systems, and CI/CD pipelines to enable end-to-end testing across flight code, ML models, simulation, and hardware-in-the-loop environments.

The role requires deep Python automation, distributed-systems testing, and leadership across cross-functional teams to drive robust validation from data to deployment and monitoring.

Qualifications

Responsibilities

• Own the technical strategy, architecture, and roadmap for automated testing, verification, and MLOps quality across the Hivemind ecosystem.
• Design and maintain scalable test frameworks for autonomy software, backend services, APIs, operator-facing applications, and distributed hardware environments.
• Lead functional, integration, regression, system, performance, reliability, and end-to-end testing across simulation, edge-compute, software-in-the-loop, and hardware-in-the-loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness, regression, latency, resource utilization, and system integration.
• Establish CI/CD and continuous training workflows that provide versioning and traceability for datasets, models, configurations, evaluation results, and deployment artifacts.
• Develop scenario-based validation for autonomy models, including edge cases, degraded sensing or communications, distribution shifts, and representative mission conditions.
• Create observability, analytics, and failure-triage capabilities for software behavior, model and data drift, inference health, test results, and production performance.
• Build Python automation that improves test execution, parallelization, reporting, environment setup, experiment comparison, and developer productivity.
• Create test harnesses, simulators, stubs, mocks, and synthetic data capabilities that improve system testability and coverage.
• Collaborate with software, autonomy, machine learning, data, simulation, and systems engineers to define verification strategies and improve designs before implementation.
• Develop and govern AI-assisted engineering workflows using coding agents and LLM-based tools for test generation, log analysis, debugging, and failure triage while maintaining security, reproducibility, and traceability.

Job description

Shield AI is seeking a Staff Automated Test Engineer to lead automated verification for its Hivemind autonomy platform. You will shape test architecture, MLOps quality systems, and CI/CD pipelines to enable end-to-end testing across flight code, ML models, simulation, and hardware-in-the-loop environments.

The role requires deep Python automation, distributed-systems testing, and leadership across cross-functional teams to drive robust validation from data to deployment and monitoring.