AI Quality Engineer: End-to-End ML & Data Validation

Cavendish Professionals; Town of Italy (NY); On-site; USD 95,000 - 120,000; Full time

Job summary

A leading recruitment agency is seeking an AI QA Engineer to define and lead the QA strategy for AI and data-driven systems. The role requires 3-6 years of experience in QA or SDET, proficiency in Python scripting, and a strong understanding of ML evaluation tools. Responsibilities include creating test plans, conducting scenario-based tests, and integrating tests into CI/CD pipelines. The position is located in New York and promotes equal opportunity employment.

Qualifications

• 3–6+ years of experience in QA or SDET, ideally with AI/ML testing.
• Capability to evaluate data correctness and retrieval accuracy.
• Experience in designing test plans for non-functional and behavioral requirements.

Responsibilities

• Define and own the end-to-end QA strategy across AI components.
• Conduct scenario-based tests and regression tests.
• Integrate automated tests into CI/CD pipelines.

Skills

Python scripting skills

ML evaluation tools

Experience with LLM/RAG testing

Knowledge of vector databases

Understanding of CI/CD pipelines

Attention to edge cases

Excellent communication skills

Job description

A leading recruitment agency is seeking an AI QA Engineer to define and lead the QA strategy for AI and data-driven systems. The role requires 3-6 years of experience in QA or SDET, proficiency in Python scripting, and a strong understanding of ML evaluation tools. Responsibilities include creating test plans, conducting scenario-based tests, and integrating tests into CI/CD pipelines. The position is located in New York and promotes equal opportunity employment.