Senior AI QA Engineer: Annotation & Evaluation Lead

BrightClaim; Greater London; On-site; GBP 70,000 - 110,000; Full time

Job summary

Genpact in London is seeking a Sr. Engineer for Testing and Quality Assurance to lead AI annotation, evaluation, and validation activities. You will ensure outputs meet quality standards and align with predefined golden questions and matrices.

You will collaborate with AI teams to define evaluation frameworks, maintain annotation schemas, and drive improvements across multiple AI use cases. The role involves SQL-based validation and quality governance in a fast-moving innovation environment.

Qualifications

• Experience in AI annotation, evaluation and QA testing.
• Ability to design and apply evaluation guidelines and QA standards.
• Strong collaboration with AI teams to govern schemas and quality metrics.

Responsibilities

• Execute annotation, evaluation, and validation activities for AI-powered solutions using established guidelines and quality standards.
• Validate AI-generated outputs against predefined golden questions and expected results.
• Create, maintain, and review annotations to support AI model evaluation and continuous improvement.
• Test and evaluate Databricks Genie responses for accuracy, completeness, relevance, and consistency.
• Write and execute SQL queries against underlying data sources to validate answers generated from golden questions and Databricks Genie outputs.
• Investigate discrepancies between expected results, source data, and AI-generated responses, and document findings.

Skills

Annotation

SQL queries

AI model evaluation

Quality assurance

Collaboration

Tools

Databricks Genie

Job description

Genpact in London is seeking a Sr. Engineer for Testing and Quality Assurance to lead AI annotation, evaluation, and validation activities. You will ensure outputs meet quality standards and align with predefined golden questions and matrices.

You will collaborate with AI teams to define evaluation frameworks, maintain annotation schemas, and drive improvements across multiple AI use cases. The role involves SQL-based validation and quality governance in a fast-moving innovation environment.