Senior AI Evals Engineer — Quality & Evaluation Lead

Alpha10x

Doha

On-site

QAR 360,000 - 600,000

Full time

12 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

ALPHA10X in Doha seeks a Senior AI Evals Engineer to make AI quality measurable. You own evaluation across the platform: datasets, metrics, judges and pipelines, converting results into decisions that improve models and deployments.

Reporting to the Head of Engineering, you shape evaluation standards, build scalable pipelines, and collaborate with AI engineers, data scientists, Product and Solutions to raise the bar across the company.

Qualifications

  • Master’s degree in computer science or engineering.
  • 7+ years in ML engineering, applied science, or software with AI in production.
  • Hands-on ownership of evaluation for LLM or ML systems in production.
  • Strong knowledge of offline and online evaluation, A/B testing and related methods.

Responsibilities

  • Define evaluation strategy and frameworks for AI systems, including benchmarking.
  • Design and maintain evaluation datasets reflecting real usage and edge cases.
  • Integrate evals into delivery workflow and monitor production for quality drift.
  • Collaborate with AI engineers, data scientists, Product and Solutions; report clearly.

Skills

Python
ML engineering
Evaluation pipelines
Experiment design

Education

Master’s degree in computer science or engineering

Job description

ALPHA10X in Doha seeks a Senior AI Evals Engineer to make AI quality measurable. You own evaluation across the platform: datasets, metrics, judges and pipelines, converting results into decisions that improve models and deployments.

Reporting to the Head of Engineering, you shape evaluation standards, build scalable pipelines, and collaborate with AI engineers, data scientists, Product and Solutions to raise the bar across the company.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Engineer - Production-Ready Agentic Systems Doha
Senior AI Engineer - Production-Ready Agentic Systems Doha

Alpha10x • Doha

On-site
QAR 300,000 - 540,000
Senior AI Evals Engineer
Senior AI Evals Engineer

Alpha10x • Doha

On-site
QAR 360,000 - 600,000
Doha Senior Solutions Architect, AI Platform Lead
Doha Senior Solutions Architect, AI Platform Lead

Alpha10x • Doha

On-site
QAR 546,000 - 764,000
Doha-Based Senior Platform Engineer: Build the AI Platform
Doha-Based Senior Platform Engineer: Build the AI Platform

Alpha10x • Doha

On-site
QAR 150,000 - 230,000
Doha Senior Data Engineer — AI Pipelines for Fintech
Doha Senior Data Engineer — AI Pipelines for Fintech

Alpha10x • Doha

On-site
QAR 180,000 - 300,000
Senior QA Lead: AI-Driven Quality & CX Strategy
Senior QA Lead: AI-Driven Quality & CX Strategy

Snoonu • Lusail

On-site
QAR 240,000 - 420,000
Global Vibes
Learning budgets
Flexible time off
+2
Senior AI Architect: Production ML & RAG Systems Lead
Senior AI Architect: Production ML & RAG Systems Lead

Cplusoft • Doha

On-site
QAR 400,000 - 800,000
AI/ML Systems Lead — Architect & Scale Data Platforms
AI/ML Systems Lead — Architect & Scale Data Platforms

Keka Technologies Private Limited • Qatar

Remote
QAR 320,000 - 540,000
Global team
Learning budget
Flexible time off
+1
Artificial Intelligence Engineer
Artificial Intelligence Engineer

NEX Solutions • Doha

On-site
QAR 180,000 - 240,000
AI QA Architect: End-to-End LLM Testing & Automation
AI QA Architect: End-to-End LLM Testing & Automation

Arabic.AI • Doha

On-site
QAR 180,000 - 240,000
Top locations