Senior AI Evals Engineer

Alpha10x

Doha

Presencial

QAR 360.000 - 600.000

Jornada completa

hace 11 horas
Sé de los primeros/as/es en solicitar esta vacante

Consigue una respuesta de este empleador

Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.

Supera los filtros ATS

Descripción de la vacante

ALPHA10X in Doha seeks a Senior AI Evals Engineer to make AI quality measurable. You own evaluation across the platform: datasets, metrics, judges and pipelines, converting results into decisions that improve models and deployments.

Reporting to the Head of Engineering, you shape evaluation standards, build scalable pipelines, and collaborate with AI engineers, data scientists, Product and Solutions to raise the bar across the company.

Formación

  • Master’s degree in computer science or engineering.
  • 7+ years in ML engineering, applied science, or software with AI in production.
  • Hands-on ownership of evaluation for LLM or ML systems in production.
  • Strong knowledge of offline and online evaluation, A/B testing and related methods.

Responsabilidades

  • Define evaluation strategy and frameworks for AI systems, including benchmarking.
  • Design and maintain evaluation datasets reflecting real usage and edge cases.
  • Integrate evals into delivery workflow and monitor production for quality drift.
  • Collaborate with AI engineers, data scientists, Product and Solutions; report clearly.

Conocimientos

Python
ML engineering
Evaluation pipelines
Experiment design

Educación

Master’s degree in computer science or engineering

Descripción del empleo

We are hiring a Senior AI Evals Engineer to make the quality of our AI systems measurable.

ALPHA10X is an award-winning, AI-native Decision Intelligence platform built for private markets - a market projected to exceed $65 trillion in AUM by 2032, but one in which investment decisions remain constrained by fragmented data, subjective judgment and limited predictive infrastructure. At its core is ALPHAcodex, a governed reasoning system that combines data, inference, institutional memory and continuous learning to generate quantitative signals and probabilistic foresight. The platform is designed to improve underwriting and capital allocation over time: each decision becomes institutional memory, strengthening the intelligence applied to subsequent decisions and creating the potential for a compounding advantage in judgment, conviction and investment performance.

Founded in France, ALPHA10X has progressively shifted its center of gravity to the Middle East as the region has become increasingly important to global capital allocation. The company is establishing its global headquarters and principal operating hub in Doha, providing a strategic base for product development, commercial expansion and global growth.

ALPHA10X has raised $25 million in seed capital from prominent investors and academics and completed more than 70 engagements across 11 countries. The company is now entering its next phase of scale, strengthening leadership, accelerating R&D, building its Doha organization and expanding institutional market penetration ahead of a planned Series A financing.

We are hiring a Senior AI Evals Engineer to make the quality of our AI systems measurable. Our agents, retrieval layer, and scoring models inform real investment decisions, so “it looks good” is not good enough. Your job is to tell a real improvement apart from a plausible-looking one, and to give the team the tools to ship model, prompt, and retrieval changes on evidence rather than intuition.

Reporting to the Head of Engineering, you will own evaluation across the AI platform: the datasets, metrics, judges, and pipelines that tell us whether our systems are getting better, and why they fail when they do.

This is an individual contributor role. You will not manage a team; you will shape how the team works through the evaluation standards you set, the evidence you bring to decisions, and the rigor you model in every review.

The successful candidate will be someone who takes initiative, stays curious and has an unrelenting drive to push the boundaries of AI and software engineering. They will be adaptive and introspective: willing to learn, guide, lead and follow, with the low ego of someone for whom the outcome matters more than who gets the credit.

Key Responsibilities
  • Evaluation Strategy & Frameworks: Define how we judge quality for each AI system: agents, search, extraction, and scoring. Build the evaluation framework and harness the whole team uses, so benchmarking our systems is as routine as running a test suite.
  • Datasets & Ground Truth: Design and maintain evaluation datasets that reflect real usage: golden sets, adversarial and edge cases, and regression suites. Work with investment and domain experts to define what a “good answer” means compared to a merely plausible one.
  • Release Gating & Online Evaluation: Integrate evals into our delivery workflow and release assessment. Run tests and online monitoring of production and catch quality drifts early.
  • Cross-functional Collaboration: Work closely with AI engineers, data scientists, Product, and Solutions. Make quality legible to non-specialists through clear reports and dashboards and raise the evaluation bar across the company through collaboration and example.
What We Are Looking For

This role requires a combination of evaluation expertise, statistical rigor, and strong engineering. We are looking for someone who:

  • Has a Master’s degree (minimum) in computer science or engineering.
  • Has 7+ years in ML engineering, applied science, or software engineering with AI, including hands-on ownership of evaluation for LLM or ML systems in production.
  • Has deep, practical knowledge of offline and online evaluation (dataset design, human annotation and inter-rater agreement, LLM-as-judge design and validation, and A/B testing) and knows the limits and failure modes of each.
  • Has a solid grounding in experimental design, significance testing, confidence intervals, and sampling, enough to say when a difference is real and when it is noise.
  • Has a practical understanding of tool-using agents and prompt engineering, and of where each tends to fail.
  • Is an expert in Python, writes clean, testable code, and is comfortable building eval pipelines that run reliably at scale.
  • Has an ownership mindset: takes a problem end to end, including the unglamorous parts.
  • Communicates clearly and concisely and is comfortable working with researchers and engineers alike.
  • Brings pragmatic problem-solving and a bias for action in a fast-moving environment.
  • Is curious about business domains and quickly develops an instinct for what good looks like.
  • Mentors through example, making the people around them better without needing authority to do it.
Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Senior AI Engineer
Senior AI Engineer

Alpha10x • Doha

Presencial
QAR 300.000 - 540.000
Senior FDE Engineer
Senior FDE Engineer

Alpha10x • Doha

Presencial
QAR 300.000 - 450.000
Senior AI Evals Engineer — Quality & Evaluation Lead
Senior AI Evals Engineer — Quality & Evaluation Lead

Alpha10x • Doha

Presencial
QAR 360.000 - 600.000
Senior Data Engineer
Senior Data Engineer

Alpha10x • Doha

Presencial
QAR 180.000 - 300.000
Senior Solutions Architect
Senior Solutions Architect

Alpha10x • Doha

Presencial
QAR 546.000 - 764.000
Senior AI Engineer
Senior AI Engineer

Cplusoft • Doha

Presencial
QAR 400.000 - 800.000
Senior Software Engineer, AI Operations
Senior Software Engineer, AI Operations

Scale AI, Inc. • Qatar

Presencial
QAR 300.000 - 360.000
Agent Evaluation Engineer
Agent Evaluation Engineer

Employment • Doha

A distancia
QAR 207.000 - 295.000
Staff / Principal Engineer - AI-Native Engineering
Staff / Principal Engineer - AI-Native Engineering

TechTree • Doha

Híbrido
QAR 600.000 - 1.000.000
Hybrid work model
Senior AI Architect for Enterprise Solutions
Senior AI Architect for Enterprise Solutions

Black Grey • Qatar

Presencial
QAR 180.000 - 240.000
Ubicaciones