AI Research Engineer - Coding Agents Evaluation (Remote)

Joblet-AI

United States

Remote

GBP 151,000 - 197,000

Full time

44 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity compensation
Performance-based bonuses
Health-insurance reimbursement
401(K) plan with company match
Paid time off

Job summary

micro1 is seeking a Member of Technical Staff to advance evaluation and development of frontier coding agents. You will design benchmarks, methodologies, and data systems that shape how next-generation coding models are measured and improved.

You will lead end-to-end research on measuring coding model performance, build datasets and evaluation protocols, and collaborate with researchers, engineers, and applied AI teams to drive rigorous experimentation and reliable insights.

Qualifications

  • Strong software engineering background with Python, C++, or comparable programming languages.
  • Experience designing, reviewing, or validating technical assessments, benchmarks, coding tasks, or evaluation methodologies.
  • Familiarity with large language models, coding agents, reinforcement learning, model evaluation, or related AI systems.

Responsibilities

  • Design and own evaluation frameworks for coding agents, including benchmark specifications, scoring methodologies, rubrics, and quality standards.
  • Lead end-to-end research initiatives focused on measuring and improving coding model performance across diverse software engineering tasks.
  • Develop high-quality datasets, golden examples, and evaluation protocols that enable reliable assessment of frontier coding systems.
  • Analyze model behavior and failure modes, identifying systematic weaknesses and translating findings into actionable improvements for training and evaluation.
  • Build tooling and infrastructure that support large-scale experimentation, data generation, review workflows, and evaluation pipelines.
  • Establish best practices for coding-agent assessment, ensuring methodological rigor, reproducibility, and measurement quality.
  • Partner closely with researchers, engineers, and applied AI teams to design experiments and evaluate emerging model capabilities.
  • Contribute to technical reports, benchmark studies, and client-facing research initiatives that communicate model performance and insights.

Skills

Python
C++
Software engineering

Job description

micro1 is seeking a Member of Technical Staff to advance evaluation and development of frontier coding agents. You will design benchmarks, methodologies, and data systems that shape how next-generation coding models are measured and improved.

You will lead end-to-end research on measuring coding model performance, build datasets and evaluation protocols, and collaborate with researchers, engineers, and applied AI teams to drive rigorous experimentation and reliable insights.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Coding Research Evaluation Engineer
Coding Research Evaluation Engineer

Remotebridge • United States

Remote
USD 200,000 - 260,000
Equity compensation
Bonus eligibility
Health-insurance reimbursement
+2
Remote Staff Scientist, Coding Agent Evaluation & Benchmarks
Remote Staff Scientist, Coding Agent Evaluation & Benchmarks

Remotebridge • United States

Remote
USD 200,000 - 260,000
Equity compensation
Performance-based bonuses
Health-insurance premium reimbursement
+2
Remote Staff Engineer - Coding Agent Evaluation & Research
Remote Staff Engineer - Coding Agent Evaluation & Research

micro1 Inc. • United States

Remote
USD 200,000 - 260,000
Equity compensation
Health insurance premium reimbursement
401(k) with company match
+1
Member of Technical Staff, Coding Agent Research & Evaluations
Member of Technical Staff, Coding Agent Research & Evaluations

Frontier AI Lab • United States

Remote
USD 200,000 - 260,000
Equity compensation
Performance-based bonuses
Health-insurance premium reimbursement
+2
Member of Technical Staff, Coding Agent Research & Evaluations
Member of Technical Staff, Coding Agent Research & Evaluations

micro1 Inc. • United States

Remote
USD 200,000 - 260,000
Equity compensation
Health insurance premium reimbursement
401(k) with company match
+1
Member of Technical Staff, Coding Research
Member of Technical Staff, Coding Research

Remotebridge • United States

Remote
USD 200,000 - 260,000
Equity compensation
Bonus eligibility
Health-insurance reimbursement
+2
AI Research Engineer (Remote)
AI Research Engineer (Remote)

Joblet-AI • United States

Remote
GBP 151,000 - 197,000
Equity compensation
Performance-based bonuses
Health-insurance reimbursement
+2
Member of Technical Staff, Coding Agent Research & Evaluations
Member of Technical Staff, Coding Agent Research & Evaluations

Confidential • United States

Remote
USD 200,000 - 260,000
Equity compensation
Performance-based bonuses
Health-insurance premium reimbursement
+2
Remote Lead, Coding-Agent Evaluation & Benchmarking
Remote Lead, Coding-Agent Evaluation & Benchmarking

Confidential • United States

Remote
USD 200,000 - 260,000
Equity compensation
Performance-based bonuses
Health-insurance premium reimbursement
+2
Senior Software Engineer, AI Evaluation & Coding Agents
Senior Software Engineer, AI Evaluation & Coding Agents

24 Mag • New York (NY)

Remote
USD 138,000 - 207,000