Automation Test Engineer

Jobtailor; Deutschland; Remote; EUR 65.000 - 90.000; Vollzeit

Zusammenfassung

Jobtailor is seeking a senior QA automation engineer to support an AI/ML platform for government-grade applications. You will design automated tests for web apps, APIs, and AI features, and help build pipelines for model validation, bias detection, and drift monitoring.

Collaborate with AI/ML engineers, implement quality gates in CI/CD, mentor juniors, and contribute to ethical AI standards in a DevSecOps setting.

Qualifikationen

• 5+ years in software QA/automation with AI/ML testing focus.
• Hands-on with AI model testing, including datasets and explainability.
• Experience with Agile DevSecOps and CI/CD tooling.

Aufgaben

• Support development of an AI/ML platform and test AI-driven features.
• Design and maintain scalable automated test frameworks for web apps and APIs.
• Develop automated pipelines for model validation, bias detection, and drift monitoring.
• Define and execute functional, regression, performance test plans.
• Collaborate with AI/ML engineers to create test datasets and simulate real user behavior.
• Implement CI/CD-integrated quality gates for AI release reliability and fairness.
• Mentor junior engineers and promote best practices.

Kenntnisse

AI/ML Product Testing

Automated Testing Frameworks

Model Validation Metrics

CI/CD Tooling

Collaboration Skills

Tools

MLflow

TensorBoard

Jenkins

GitHub Actions

GitLab

Selenium

Cypress

Playwright

Python

Jobbeschreibung

• Support development of an AI/ML platform leveraging generative AI to relieve user burden across the Centers for Medicare and Medicaid Services (CMS)
• Design and maintain scalable automated test frameworks for web applications, APIs, and AI-driven features
• Develop automated pipelines for model validation, bias detection, and drift monitoring in ML-based systems
• Define and execute functional, regression, performance, and AI-specific test plans
• Partner with AI/ML engineers to create test datasets and simulate real-world user behavior and edge cases
• Implement CI/CD-integrated quality gates for AI release reliability and fairness metrics
• Mentor junior engineers and establish best practices in intelligent automation and AI testing
• Advocate for ethical AI standards and contribute to responsible AI discussions

Requirements

• 5+ years of experience in software QA/automation engineering, with at least 1 year focused on AI/ML product testing
• Strong proficiency in at least one automation framework (e.g., Selenium, Cypress, Playwright) and scripting language (Python preferred)
• Hands-on experience with testing AI models, including validation metrics, dataset versioning, and model explainability
• Familiarity with tools like MLflow, TensorBoard, or similar for AI observability
• Understanding of model bias, fairness audits, and compliance considerations in government or regulated environments
• Experience working in Agile DevSecOps environments with modern CI/CD tooling (Jenkins, GitHub Actions, GitLab, etc.)
• Ability to obtain and maintain a Public Trust security clearance
• Acceptable results of a background investigation
• Strong collaboration skills with clear communication across engineering, data science, and product teams

Core Competencies

Demonstrates expertise in AI/ML product testing, including automated testing frameworks, model validation, and ethical AI standards. Proficient in CI/CD practices and collaboration across engineering and data science teams.

Highest-signal resume keywords

• AI/ML Product Testing
• Automated Testing Frameworks
• Model Validation Metrics
• CI/CD Tooling
• Collaboration Skills

Hard Skills

• Software QA Engineering
• Automation Engineering
• Selenium
• Cypress
• Playwright
• Python
• Model Explainability
• Dataset Versioning
• Bias Detection
• Drift Monitoring

Soft Skills

• Collaboration
• Clear Communication
• Mentoring

Certifications & Qualifications

• Public Trust Security Clearance

Industry Keywords

• Agile DevSecOps
• Compliance Considerations
• Ethical AI Standards
• Fairness Audits

Tools & Technologies

• MLflow
• TensorBoard
• Jenkins
• GitHub Actions
• GitLab