AI Quality Engineer

TSR Consulting; Irving (TX); On-site; USD 90,000 - 130,000; Full time

Job summary

TSR Consulting seeks an AI Quality Engineer to join our team in Irving, TX. You will lead quality assurance for AI/ML models, including Generative AI, and design comprehensive testing strategies that ensure reliability, fairness, and performance of AI systems in production.

You will develop test plans, create edge-case test cases, and drive automation across AI pipelines and model validation. Collaboration with data scientists and cross-functional teams is essential to embed quality throughout

Qualifications

• Strong experience in AI and ML quality assurance.
• Experience with test automation and model validation.
• Ability to design comprehensive testing strategies for AI systems.

Responsibilities

• Quality assurance strategy and standards for AI/ML models, including Generative AI.
• Develop test strategies and plans covering objectives, scope, resources, and schedules.
• Design test cases across functional, non-functional, performance, regression, and adversarial testing.
• Assess AI robustness and fairness against edge cases and unexpected scenarios.
• Conduct performance and scalability testing for model inference and resource use.
• Collaborate with data scientists on data quality and bias assessment.
• Develop automation for AI/ML pipelines, model validation, and system integration.
• Manage defects from identification to resolution.
• Define metrics (accuracy, precision, recall, F1, latency) and report to stakeholders.
• Embed quality throughout the AI lifecycle with cross-functional teams.
• Pursue continuous improvements from advances in AI testing methodologies.

Skills

AI quality assurance

Test automation

Model validation

Performance testing

Job description

We are seeking a AI Quality Engineer to join our dynamic team. The ideal candidate will have strong experience in AI and ML quality assurance, test automation, and model validation and a proven ability to design and execute comprehensive testing strategies that ensure reliability, fairness, and performance of AI systems in production.

Responsibilities:

• Quality assurance strategy and standards for AI and ML models, including Generative AI solutions and autonomous agents.
• Test strategy and plan development covering objectives, scope, resources, and schedules for ethical and robust AI evaluation.
• Test case design and execution across functional, non-functional, performance, stress, regression, and adversarial testing.
• AI robustness and functionality testing against edge cases and unexpected scenarios to ensure consistent and accurate results.
• Performance and scalability testing of model inference speed, resource utilization, and behavior under load.
• Data quality and model bias assessment in partnership with data scientists to identify and mitigate fairness issues.
• Automation development for AI/ML pipelines, model validation, and system integration.
• Defect management including identification, documentation, prioritization, and coordination for resolution.
• Metric definition and reporting of accuracy, precision, recall, F1-score, latency, and reliability to stakeholders.
• Collaboration with cross-functional teams to embed quality throughout the AI lifecycle.
• Continuous improvement through adoption of advancements in AI testing methodologies, tools, and practices.