Staff Engineer, Test Automation - San Mateo (R5792)

F7d8749a; San Mateo (CA); Sur place; USD 180 000 - 280 000; Plein temps

Résumé du poste

Shield AI's Hivemind Software Engineering Integration and Test team seeks a Staff Automated Test Engineer to lead automated verification across our autonomy platform. You will define test architecture, validation infrastructure, and CI/CD pipelines for deploying resilient software in unmanned aircraft and robotic systems.

You will work across production flight code, ML models, simulation environments, mission planning, and data pipelines, mentoring teams and shaping verification standards in a

Qualifications

• 8+ years in software engineering, test infra, or MLops.
• 5+ years building scalable automation in Python.
• Experience validating ML systems end-to-end.
• Experience with GPU workloads in Kubernetes.
• Understanding ML quality risks and data leakage.
• Experience designing CI/CD and MLOps workflows.

Responsabilités

• Own the strategy, architecture, and roadmap for automated testing and MLOps quality across the Hivemind ecosystem.
• Design scalable test frameworks for autonomy software, backend services, APIs, operator-facing applications, and distributed hardware environments.
• Lead functional, integration, regression, system, performance, reliability, and end-to-end testing across simulation, edge-compute, software-in-the-loop, and hardware-in-the-loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness, regression, latency, resource utilization, and system integration.
• Establish CI/CD and continuous training workflows that provide versioning and traceability for datasets, models, configurations, evaluation results, and deployment artifacts.
• Develop scenario-based validation for autonomy models, including edge cases, degraded sensing or communications, distribution shifts, and representative mission conditions.
• Create observability, analytics, and failure-triage capabilities for software behavior, model and data drift, inference health, test results, and production performance.
• Build Python automation that improves test execution, parallelization, reporting, environment setup, experiment comparison, and developer productivity.
• Create test harnesses, simulators, stubs, mocks, and synthetic data capabilities that improve system testability and coverage.
• Collaborate with software, autonomy, machine learning, data, simulation, and systems engineers to define verification strategies and improve designs before implementation.
• Develop and govern AI-assisted engineering workflows using coding agents and LLM-based tools for test generation, log analysis, debugging, and failure triage while maintaining security, reproducibility, and traceability.

Connaissances

Python automation

MLOps

Test automation

Software testing

Distributed systems

Linux

CI/CD

Outils

Kubernetes

CI/CD pipelines

Python

Description du poste

Shield AI is a venture-backed defense-tech company with the mission of protecting service members and civilians with intelligent systems. Its products include Hivemind autonomy software, V-BAT and X-BAT aircraft, and Aechelon simulation and synthetic reality technologies. With offices and facilities across the U.S., Europe, the Middle East, and Asia-Pacific, Shield AI’s technology actively supports operations worldwide. For more information, visit www.shield.ai https://shield.ai/. Follow Shield AI on LinkedIn https://www.linkedin.com/company/shield-ai, X https://x.com/shieldaitech, Instagram https://www.instagram.com/shield_ai, and YouTube https://www.youtube.com/@Shield_AI.

The Hivemind Software Engineering Integration and Test team is seeking a Staff Automated Test Engineer to provide technical leadership for automated verification across our next-generation autonomy platform. You will define and evolve test architecture, validation infrastructure, MLOps quality systems, and CI/CD pipelines that enable the Hivemind software ecosystem to create, test, and deploy resilient autonomy capabilities for unmanned aircraft and robotic platforms operating in complex, contested, and GPS-denied environments. You will work across production flight code, machine learning models, simulation and synthetic environments, mission planning and orchestration systems, operator-facing Ground Control Station applications, telemetry and data pipelines, cloud-native developer infrastructure, and hardware-in-the-loop systems. In this hands‑on Staff role, you will lead complex, cross‑functional initiatives, establish automation and verification standards, identify systemic quality risks, and develop scalable test infrastructure. The ideal candidate combines deep Python automation and distributed-systems testing experience with technical leadership and experience validating machine learning systems throughout the data, training, evaluation, deployment, and monitoring lifecycle.

What you'll do:

• Own the technical strategy, architecture, and roadmap for automated testing, verification, and MLOps quality across the Hivemind ecosystem.
• Design and maintain scalable test frameworks for autonomy software, backend services, APIs, operator‑facing applications, and distributed hardware environments.
• Lead functional, integration, regression, system, performance, reliability, and end‑to‑end testing across simulation, edge‑compute, software‑in‑the‑loop, and hardware‑in‑the‑loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness, regression, latency, resource utilization, and system integration.
• Establish CI/CD and continuous training workflows that provide versioning and traceability for datasets, models, configurations, evaluation results, and deployment artifacts.
• Develop scenario‑based validation for autonomy models, including edge cases, degraded sensing or communications, distribution shifts, and representative mission conditions.
• Create observability, analytics, and failure‑triage capabilities for software behavior, model and data drift, inference health, test results, and production performance.
• Build Python automation that improves test execution, parallelization, reporting, environment setup, experiment comparison, and developer productivity.
• Create test harnesses, simulators, stubs, mocks, and synthetic data capabilities that improve system testability and coverage.
• Collaborate with software, autonomy, machine learning, data, simulation, and systems engineers to define verification strategies and improve designs before implementation.
• Develop and govern AI‑assisted engineering workflows using coding agents and LLM‑based tools for test generation, log analysis, debugging, and failure triage while maintaining security, reproducibility, and traceability.

Required qualifications:

• Typically 8+ years of relevant experience in software engineering, test infrastructure, developer tooling, MLOps, systems integration, or systems verification, or an equivalent combination of experience and demonstrated impact.
• 5+ years of experience building scalable automation frameworks or developer tooling in Python.
• Demonstrated success designing test, CI/CD, or MLOps infrastructure used across multiple engineering teams.
• Experience validating machine learning systems across the data, training, evaluation, packaging, deployment, and monitoring lifecycle.
• Understanding of ML quality risks such as data leakage, training‑serving skew, nondeterminism, distribution shift, drift, model regression, and statistical acceptance criteria.
• Experience defining model‑performance baselines, automated evaluation suites, release thresholds, and candidate‑to‑production comparison workflows.
• Experience testing GPU‑accelerated infrastructure and workloads, including GPU scheduling, allocation, utilization, and resource contention in Kubernetes environments.
• Experience with performance benchmarking, profiling, and observability for GPU workloads, including identifying compute, memory, storage, networking, and data‑loading bottlenecks.
• Experience validating multi‑tenant Kubernetes environments, including RBAC, resource quotas, workload isolation, and sche