AI Coding Agent Evaluator — Design & Validate Tasks

Employment

Doha

Remote

QAR 207,000 - 295,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Employment seeks experienced software developers and testers to help build a realistic developer environment and evaluation tasks for AI coding agents. You will craft prompts, define success criteria, and write tests that accept multiple valid approaches.

This project uses Python/FastAPI, TypeScript/React, Docker, PostgreSQL, Kafka and Redis; collaboration with a remote team is required and English proficiency (B2+) is essential.

Qualifications

  • 5+ years of professional software development experience.
  • Strong hands-on with Python (FastAPI) and JS/TS (React).
  • Experience with Docker, PostgreSQL, Kafka and Redis.
  • Proven testing experience (functional/integration tests) and English proficiency (B2+).

Responsibilities

  • Create realistic developer environments including codebase, infra, tickets and docs.
  • Design tasks from intermediate states and define what ‘solved’ means.
  • Write tests that accept multiple valid approaches and reject incorrect ones.
  • Iterate on tasks based on QA feedback to ensure fairness and robustness.

Skills

5+ years software development
Python (FastAPI)
JavaScript/TypeScript (React)
Docker
PostgreSQL
Kafka
Redis
Testing experience (functional/integr.
English proficiency (B2+)

Tools

Docker
PostgreSQL
Kafka
Redis
React
FastAPI

Job description

Employment seeks experienced software developers and testers to help build a realistic developer environment and evaluation tasks for AI coding agents. You will craft prompts, define success criteria, and write tests that accept multiple valid approaches.

This project uses Python/FastAPI, TypeScript/React, Docker, PostgreSQL, Kafka and Redis; collaboration with a remote team is required and English proficiency (B2+) is essential.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Agent Evaluation Engineer
Agent Evaluation Engineer

Employment • Doha

Remote
QAR 207,000 - 295,000
Forward Deployed Engineer - Doha
Forward Deployed Engineer - Doha

present of AI • Doha

On-site
QAR 70,000 - 110,000
Freelance Agent Evaluation Engineer
Freelance Agent Evaluation Engineer

Employment • Doha

On-site
QAR 207,000 - 295,000
Project-based engagement
Senior AI Evals Engineer
Senior AI Evals Engineer

Alpha10x • Doha

On-site
QAR 360,000 - 600,000
AI Software Engineer - Full Stack Developer (Java, React)
AI Software Engineer - Full Stack Developer (Java, React)

EPAM Systems • Qatar

On-site
QAR 240,000 - 360,000
Senior QA Engineer - AI Products (6 Months Contract)
Senior QA Engineer - AI Products (6 Months Contract)

Arabic.AI • Doha

On-site
QAR 180,000 - 240,000
Arabic Language Expert
Arabic Language Expert

Employment • Doha

Remote
QAR 125,000 - 201,000
Generative AI Workflow & Agent Developer
Generative AI Workflow & Agent Developer

Qatar Insurance Group • Doha

On-site
QAR 180,000 - 320,000
Senior AI Engineer
Senior AI Engineer

Cplusoft • Doha

On-site
QAR 400,000 - 800,000
Senior AI Engineer
Senior AI Engineer

Alpha10x • Doha

On-site
QAR 300,000 - 540,000
Top locations