AI Validation Engineer

Stellantis; Auburn Hills (MI); On-site; USD 120,000 - 190,000; Full time

Job summary

Stellantis in Michigan is seeking a Senior AI Validation Engineer to design testing strategies and automated pipelines ensuring the quality, safety, and reliability of AI-driven features across vehicle platforms. You will bridge AI/ML engineering and automotive system validation, building tools, datasets, and evaluation methods for robust automotive AI solutions.

The role involves collaboration with cross-functional teams in Agile sprints, implementing regression testing for ML updates, and

Qualifications

• Bachelor's degree in CS, ML, Data Science, or related field.
• Minimum 5 years in ML/AI development with 2+ years in model evaluation or validation.
• Strong Python skills and experience with testing frameworks.
• Hands-on evaluation of deep learning models, including metrics design and dataset curation.
• Experience with LLM evaluation techniques (BLEU, ROUGE, human-in-the-loop).
• Familiarity with ML experiment tracking and pipelines (MLflow, Weights & Biases, Kubeflow).
• Experience with CI/CD for automated test execution.
• Excellent communication and ability to translate validation results.

Responsibilities

• Design and implement validation frameworks for deep learning models in automotive systems.
• Develop automated test pipelines for LLM-based features, including hallucination detection and prompt regression testing.
• Build and curate evaluation datasets and benchmarks for automotive AI use cases.
• Create AI-assisted test generation tools from system requirements.
• Develop model monitoring and drift detection in production and test environments.
• Collaborate with system architects to integrate AI validation into CI/CD pipelines.
• Analyze test results using statistics and ML to identify root causes and trends.
• Work in cross-functional Agile teams spanning AI/ML, embedded software, and system integration.

Skills

Python

Testing frameworks

ML model evaluation

LLM evaluation

CI/CD

Data science

Team collaboration

Education

Bachelor's degree in CS/ML/EE

Tools

MLflow

Weights & Biases

Kubeflow

Jenkins

GitHub Actions

Job description

As AI-powered features become central to automotive vehicles — from ADAS perception and voice assistants to predictive diagnostics and intelligent infotainment — rigorous validation of these systems is critical for successful deployment. We are seeking a Senior AI Validation Engineer to design and implement testing strategies, frameworks, and automated pipelines that ensure the quality, safety, and reliability of deep learning and LLM-based features across vehicle platforms.

This role sits at the intersection of AI/ML engineering and automotive system validation. You will build the tools, datasets, and evaluation methodologies that enable confident delivery of AI-driven automotive solutions.

Key Responsibilities:

• Design and implement validation frameworks for deep learning models (perception, NLP, generative AI) deployed in automotive systems, covering accuracy, robustness, latency, and safety metrics.
• Develop automated test pipelines for LLM-based features, including hallucination detection, response quality evaluation, prompt regression testing, and adversarial input testing.
• Build and curate evaluation datasets and benchmarks tailored to automotive AI use cases (e.g., voice commands, diagnostic Q&A, sensor fusion outputs).
• Create AI-assisted test generation tools that leverage LLMs to automatically produce test cases, test data, and expected-result specifications from system requirements.
• Develop model monitoring and drift detection systems for AI features running in production and test environments.
• Collaborate with system architects to integrate AI model validation into existing test bench infrastructure and CI/CD pipelines.
• Implement automated regression testing for ML model updates, ensuring backward compatibility and performance parity across software releases.
• Analyze test results using statistical methods and ML techniques to identify root causes, failure patterns, and quality trends.
• Work in cross-functional Agile teams spanning AI/ML, embedded software, and system integration disciplines.

Basic Qualifications:

• Bachelor’s degree in computer science, Machine Learning, Data Science, Electrical Engineering, or a related field.
• Minimum of 5 years of experience in ML/AI development, with a minimum of 2 years focused on model evaluation, testing, or validation.
• Strong proficiency in Python and testing/automation frameworks (pytest, Robot Framework, or equivalent).
• Hands-on experience evaluating deep learning models — including metrics design, dataset curation, bias/fairness analysis, and regression testing.
• Experience with LLM evaluation techniques (BLEU, ROUGE, human-in-the-loop evaluation, LLM-as-judge approaches).
• Familiarity with ML experiment tracking and pipeline orchestration tools (MLflow, Weights & Biases, Kubeflow, or equivalent).
• Experience with CI/CD systems (Jenkins, GitLab CI, GitHub Actions) for automated test execution.
• Strong analytical and communication skills with the ability to translate AI validation results into actionable insights for engineering teams.

Preferred Qualifications:

• Master's in Computer Science, Machine Learning, or a related field.
• Experience with simulation-based testing or digital twin environments.
• Familiarity with automotive test toolchains (dSpace, Vector CANoe, NI VeriStand) is a plus but not required.
• Ability to collaborate effectively across time zones with global engineering teams.
• Knowledge of automotive safety standards (ISO 26262, SOTIF/ISO 21448) as applied to AI systems.
• Experience with adversarial robustness testing, out-of-distribution detection, or uncertainty quantification for neural networks.