Senior AI QA Engineer (Pune )

Money Forward India

Pune District

On-site

INR 2,800,000 - 4,000,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Money Forward India seeks a Senior AI QA Engineer to ensure the quality, reliability, and performance of AI-driven products and data pipelines. You will validate AI model outputs, test AI-powered features, and ensure data integrity for training and inference.

You will collaborate with engineering, data engineering, and product teams to improve AI system quality through systematic testing, data validation, and automation, contributing to robust QA standards and best practices.

Qualifications

  • Bachelor’s degree in Computer Science, Software Engineering, Data Science, or related field.
  • 8+ years of experience in software QA, test automation, or data validation and 2+ years in AI & Data quality testing.
  • Experience testing APIs, web services, or distributed systems.
  • Strong knowledge of SQL and data validation techniques.
  • Strong with Python for data automation, TypeScript and/or Java for other automation.
  • Understanding of machine learning concepts and AI system behavior.
  • Experience with test automation tools.
  • Experience testing AI systems, LLM applications, or chatbots.
  • Experience with big data platforms.
  • Knowledge of prompt engineering or AI evaluation methods.
  • Experience working with cloud platforms AWS.
  • Familiarity with CI/CD and DevOps practices.
  • Generative AI: Enthusiastic user of Generative AI and advanced AI tooling to streamline workflows and significantly accelerate project delivery timelines.
  • Product Mindset: Proven track record to translate high-level product requirements into detailed requirements, and into comprehensive technical requirements through close partnership with Product Managers
  • Communication: Good verbal and written English skills, ensuring clarity and alignment within distributed, multi-national product engineering teams.

Responsibilities

  • Design and execute test strategies for AI/ML-powered applications and features.
  • Validate AI model outputs for accuracy, reliability, and consistency.
  • Perform prompt testing, response evaluation, and edge case validation for AI systems.
  • Identify issues such as hallucinations, bias, incorrect reasoning, or unstable responses.
  • Define acceptance criteria and quality benchmarks for AI-driven features.
  • Validate data pipelines (ETL/ELT) used to prepare datasets for AI models.
  • Test data ingestion, transformation, and loading processes to ensure reliability.
  • Verify data integrity between source systems, data warehouses, and AI models.
  • Detects and reports data anomalies, schema changes, missing data, or transformation errors.
  • Create data validation rules and automated data quality checks.
  • Validate training datasets and feature engineering pipelines.
  • Monitor dataset quality to prevent data drift or unexpected data changes.
  • Define and track AI model evaluation metrics such as accuracy, precision, recall, and response quality.
  • Collaborate with data scientists to analyze model performance and identify improvement areas.

Skills

AI/ML testing
SQL & dataset analysis
API testing
Data integrity verification

Education

Bachelor’s degree in Computer Science, Software Engineering, Data Science, or related field

Tools

Python
TypeScript
Java
AWS
CI/CD tools

Job description

We are seeking a Senior AI QA Engineer to ensure the quality, reliability, and performance of AI-driven products and the data pipelines that support them. This role focuses on validating AI model outputs, testing AI-powered features, and ensuring the integrity and quality of datasets and data pipelines used for training and inference.

You will work closely with engineering, data engineering, and product teams to improve the overall quality of AI systems through systematic testing, data validation, and automation.

Key Responsibilities
AI System Quality Assurance
  • Design and execute test strategies for AI/ML-powered applications and features.
  • Validate AI model outputs for accuracy, reliability, and consistency.
  • Perform prompt testing, response evaluation, and edge case validation for AI systems.
  • Identify issues such as hallucinations, bias, incorrect reasoning, or unstable responses.
  • Define acceptance criteria and quality benchmarks for AI-driven features.
  • Validate data pipelines (ETL/ELT) used to prepare datasets for AI models.
  • Test data ingestion, transformation, and loading processes to ensure reliability.
  • Verify data integrity between source systems, data warehouses, and AI models.
  • Detects and reports data anomalies, schema changes, missing data, or transformation errors.
  • Create data validation rules and automated data quality checks.
  • Validate training datasets and feature engineering pipelines.
  • Monitor dataset quality to prevent data drift or unexpected data changes.
  • Define and track AI model evaluation metrics such as accuracy, precision, recall, and response quality.
  • Collaborate with data scientists to analyze model performance and identify improvement areas.
Test Automation
  • Develop automated tests for AI APIs, workflows, and data pipelines.
  • Build automation frameworks to test AI output regression and data validation.
  • Integrate automated testing into CI/CD pipelines.
Collaboration & Process Improvement
  • Work closely with AI engineers, data engineers, product managers, and QA teams.
  • Provide feedback during AI model development and deployment cycles.
  • Contribute to building AI testing standards, data quality guidelines, and QA best practices.
Requirements
  • Bachelor’s degree in Computer Science, Software Engineering, Data Science, or related field.
  • 8+ years of experience in software QA, test automation, or data validation and 2+ years of experience in AI & Data quality testing
  • Experience testing APIs, web services, or distributed systems.
  • Strong knowledge of SQL and data validation techniques.
  • Strong with Python for data automation, typescript and or Java for other automation.
  • Understanding of machine learning concepts and AI system behavior.
  • Experience with test automation tools.
  • Experience testing AI systems, LLM applications, or chatbots.
  • Experience with big data platforms.
  • Knowledge of prompt engineering or AI evaluation methods.
  • Experience working with cloud platforms AWS
  • Familiarity with CI/CD and DevOps practices.
  • Generative AI: Enthusiastic user of Generative AI and advanced AI tooling to streamline workflows and significantly accelerate project delivery timelines.
  • Product Mindset: Proven track record to translate high-level product requirements into detailed requirements, and into comprehensive technical requirements through close partnership with Product Managers
  • Communication: Good verbal and written English skills, ensuring clarity and alignment within distributed, multi-national product engineering teams.
Key Skills
  • AI/ML testing
  • SQL and dataset analysis
  • API testing
  • Data integrity verification
Success Metrics
  • Improved AI output quality and reliability (accuracy rate, latency, token, cost,...)
  • Early detection of data pipeline or dataset issues
  • Increased automation coverage for AI and data testing
  • Reduced production issues caused by data or AI failures
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Generative AI Engineer
Generative AI Engineer

Tekskills Inc. • Pune District

On-site
INR 1,200,000 - 1,600,000
AI QA Engineer
AI QA Engineer

NewVision Software • Pune District

Hybrid
INR 1,200,000 - 1,800,000
AI QA Engineer
AI QA Engineer

Mployee.me • Delhi

On-site
INR 1,500,000 - 2,800,000
AI Senior QA
AI Senior QA

Hiver, Inc. • Bengaluru

On-site
INR 2,200,000 - 3,400,000
AI or ML Engineer - Python, QA Testing
AI or ML Engineer - Python, QA Testing

Optum • Hyderabad

On-site
INR 1,800,000 - 3,200,000
QA Engineer
QA Engineer

Weekday AI • India

Remote
INR 1,000,000 - 2,500,000
Senior QA Engineer, AI
Senior QA Engineer, AI

Intercontinental Exchange Holdings, Inc. • Hyderabad

On-site
INR 2,000,000 - 4,000,000
QA Engineer
QA Engineer

TigerHire • India

Remote
INR 1,200,000 - 2,000,000
Qa Automation Lead
Qa Automation Lead

Bounteous • Gurugram District, Bengaluru

Hybrid
INR 1,200,000 - 2,000,000
Senior QA Engineer, AI
Senior QA Engineer, AI

ICE • Hyderabad

On-site
INR 1,800,000 - 3,000,000