AI Testing & QA Lead: Annotation & Validation Architect

9752 Genpact (Uk) Limited; Greater London; Hybrid; GBP 40,000 - 60,000; Full time

Job summary

Genpact (Uk) Limited is recruiting for a fixed-term role focused on data annotation, evaluation, and quality assurance for AI-powered solutions. The role involves executing annotation, validation, and process improvements while maintaining high data quality and governance standards.

Responsibilities include validating AI outputs against golden questions, writing SQL queries for data verification, and mentoring junior annotators to ensure consistent quality across multiple AI use cases.

Qualifications

• Bachelor's degree in Computer Engineering, CS, Information Technology, or related field.
• Master's in Software Engineering preferred or equivalent.
• Advanced English language proficiency (C1).

Responsibilities

• Execute annotation, evaluation, and validation activities for AI-powered solutions using established guidelines and quality standards.
• Validate AI-generated outputs against golden questions and expected results.
• Create, maintain, and review annotations to support AI model evaluation and continuous improvement.
• Test and evaluate Databricks Genie responses for accuracy, completeness, relevance, and consistency.
• Write and execute SQL queries against underlying data sources to validate answers from golden questions and Genie outputs.
• Investigate discrepancies between expected results, source data, and AI-generated responses, documenting findings.
• Lead the design, optimization, and standardization of annotation processes and prompts across multiple AI use cases.
• Collaborate with AI teams to define governance of annotation schemas and evaluation frameworks.
• Review complex AI outputs and provide quality assessments and recommendations.
• Ensure traceability and auditability of annotation and evaluation results.
• Define reusable prompt frameworks aligned with compliance and responsible AI goals.
• Evaluate prompt performance and quality metrics to identify improvement opportunities.
• Establish quality benchmarks, calibration processes, and escalation procedures.
• Identify bias, hallucinations, and AI risks, and propose mitigation strategies.
• Support AI governance and responsible AI initiatives.
• Participate in calibration sessions to align on standards and criteria.
• Collaborate with AI engineers, data scientists, product teams, and stakeholders.
• Drive continuous process improvement and innovate with new AI techniques.
• Mentor and coach junior annotators and prompt engineers.

Skills

Data Annotation

SQL Query Writing

Education

Bachelors - Computer Engineering

Bachelors - Computer Science

Bachelors - Information Technology

Masters - Software Engineering

Tools

Databricks AI/BI Genie

Job description

Genpact (Uk) Limited is recruiting for a fixed-term role focused on data annotation, evaluation, and quality assurance for AI-powered solutions. The role involves executing annotation, validation, and process improvements while maintaining high data quality and governance standards.

Responsibilities include validating AI outputs against golden questions, writing SQL queries for data verification, and mentoring junior annotators to ensure consistent quality across multiple AI use cases.