Remote AI Coding Agent Evaluation Engineer

Joblet-AI

United Arab Emirates

Remote

AED 734,000 - 955,000

Full time

5 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity
Health insurance
Paid time off
401(k) with match
Remote-friendly

Job summary

micro1 seeks a Member of Technical Staff to advance evaluation and development of frontier coding agents. You will design benchmarks, methodologies, and data systems shaping how coding models are measured and improved.

You will lead end-to-end research, develop datasets and evaluation protocols, analyze model behavior, and build tooling for large-scale experiments. Join a remote-first team driving rigorous AI evaluation in software engineering.

Qualifications

  • Strong software engineering background with Python, C++, or comparable languages.
  • 3+ years of experience in software engineering, ML research, evaluation, or related fields.
  • Experience designing, reviewing, or validating assessments, benchmarks or evaluation methodologies.
  • Familiarity with large language models, coding agents, RL, or related AI systems.
  • Proven ability to build tooling, automate workflows, and improve technical processes.

Responsibilities

  • Design and own evaluation frameworks for coding agents, including benchmarks, scoring, rubrics, and quality standards.
  • Lead end-to-end research initiatives to measure and improve coding model performance across software tasks.
  • Develop high-quality datasets, golden examples, and evaluation protocols for reliable assessment.
  • Analyze model behavior and failure modes to identify weaknesses and propose improvements.
  • Build tooling and infrastructure to support large-scale experiments, data generation, and evaluation pipelines.
  • Establish best practices for coding-agent assessment to ensure rigor and reproducibility.
  • Collaborate with researchers, engineers, and applied AI teams to design experiments and evaluate model capabilities.
  • Contribute to technical reports, benchmark studies, and client-facing research.

Skills

Python
C++
Software Eng
ML Research
Evaluation frameworks

Job description

micro1 seeks a Member of Technical Staff to advance evaluation and development of frontier coding agents. You will design benchmarks, methodologies, and data systems shaping how coding models are measured and improved.

You will lead end-to-end research, develop datasets and evaluation protocols, analyze model behavior, and build tooling for large-scale experiments. Join a remote-first team driving rigorous AI evaluation in software engineering.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI Evaluation Engineer for Coding Agents
Remote AI Evaluation Engineer for Coding Agents

Joblet-AI • Abu Dhabi

Remote
AED 734,000 - 955,000
AI Research Engineer (Remote)
AI Research Engineer (Remote)

Joblet-AI • Abu Dhabi

Remote
AED 734,000 - 955,000
AI Research Engineer (Remote)
AI Research Engineer (Remote)

Joblet-AI • United Arab Emirates

Remote
AED 734,000 - 955,000
Equity
Health insurance
Paid time off
+2
Remote Finance AI Researcher: Evaluation & Benchmarking
Remote Finance AI Researcher: Evaluation & Benchmarking

Joblet-AI • Abu Dhabi

Remote
AED 734,000 - 918,000
Equity compensation
Bonus potential
Health-insurance premium reimbursement
+3
Junior Software Engineer (Remote)
Junior Software Engineer (Remote)

Joblet-AI • United Arab Emirates

On-site
AED 60,000 - 120,000
Remote AI-Native Fullstack Engineer
Remote AI-Native Fullstack Engineer

Joblet-AI • Abu Dhabi

Remote
AED 441,000 - 588,000
Equity compensation
Health insurance reimbursement up to 0
Paid time off
+2
Remote Forward-Deployed AI Engineer
Remote Forward-Deployed AI Engineer

Joblet-AI • Abu Dhabi

Remote
AED 661,000 - 918,000
Equity compensation
Performance-based bonuses
Remote-first workforce
+1
AI Engineer (Remote)
AI Engineer (Remote)

Joblet-AI • United Arab Emirates

Remote
AED 477,000 - 588,000
Equity compensation
Health insurance reimbursement
PTO
Fullstack Software Engineer (Remote)
Fullstack Software Engineer (Remote)

Joblet-AI • Abu Dhabi

Remote
AED 441,000 - 588,000
Equity compensation
Health insurance reimbursement up to 0
Paid time off
+2
AI Engineer
AI Engineer

Solumn AI • Dubai

Remote
AED 300,000 - 520,000
100% Remote
6-day work week
Top locations