Senior AI Evaluation Architect

Genpact; Quezon; On-site; PHP 1,200,000 - 1,800,000; Full time

Job summary

Genpact leads the AI Gigafactory and champions innovative AI solutions that transform how global enterprises operate. As Lead AI Engineer, you will shape evaluation frameworks, benchmark performance, and guide large‑scale human assessments across product teams.

You will collaborate with researchers, data scientists, and safety teams to ensure responsible, measurable outcomes. You will design robust evaluation methodologies, translate results into concrete guidance, and help steer roadmaps with

Qualifications

• Relevant years of experience in AI/ML evaluation, model testing, data science, or related technical roles.
• Strong understanding of machine learning concepts, model behavior, and evaluation metrics.
• Experience designing and running human‑in‑the‑loop evaluations or annotation workflows.
• Proficiency with Python, data analysis tools, and experiment‑tracking frameworks.
• Demonstrated ability to synthesize complex findings into clear, actionable insights.

Responsibilities

• Develop comprehensive evaluation frameworks, benchmarks, and success metrics for AI models (text, agents).
• Build scalable processes for continuous model assessment throughout the development lifecycle.
• Lead large‑scale human evaluations, including designing tasks, guidelines, and quality controls.
• Create and maintain high‑quality test datasets, including adversarial, edge‑case, and domain‑specific scenarios.
• Analyze evaluation results to identify failure patterns, risks, and opportunities for improvement.
• Translate findings into actionable recommendations for research, engineering, and product teams.
• Partner with model researchers, data scientists, product managers, and safety teams to align evaluation goals with product requirements.
• Communicate insights clearly to technical and non‑technical stakeholders, including executives.
• Influence model development roadmaps based on evaluation outcomes and risk assessments.
• Ensure evaluations meet internal safety standards and external regulatory expectations.
• Contribute to the development of responsible AI policies, red‑team strategies, and risk‑mitigation plans.

Skills

Agile Delivery

Agile Software Development Lifecycle

AI/ML Ops

AI Prompting

Data Science

Prompt Design

Prompt Engineering

Education

Bachelor’s or Master’s degree in Computer Science, ML, or related field

Job description

Genpact leads the AI Gigafactory and champions innovative AI solutions that transform how global enterprises operate. As Lead AI Engineer, you will shape evaluation frameworks, benchmark performance, and guide large‑scale human assessments across product teams.

You will collaborate with researchers, data scientists, and safety teams to ensure responsible, measurable outcomes. You will design robust evaluation methodologies, translate results into concrete guidance, and help steer roadmaps with