Member of Technical Staff: Coding Agent Evaluation

micro1; United States; Remote; USD 200,000 - 260,000; Full time

Job summary

micro1 is seeking a Member of Technical Staff to advance the evaluation and development of frontier coding agents. You will sit at the intersection of AI research, software engineering, and model evaluation, designing benchmarks, methodologies, and data systems that shape how next-generation coding models are measured and improved.

The role emphasizes rigorous experimentation, scalable tooling, and collaboration with researchers and engineers in a remote-first environment.

Qualifications

• Strong software engineering background with Python/C++ proficiency.
• 3+ years in software engineering, machine learning, AI research or related fields.
• Experience designing benchmarks, coding tasks or evaluation methodologies.
• Familiarity with large language models, coding agents, and model evaluation.
• Ability to build tooling and automate workflows for experiments.

Responsibilities

• Design and own evaluation frameworks for coding agents, including benchmarks and rubrics.
• Lead end-to-end research initiatives measuring coding model performance.
• Develop datasets, golden examples, and evaluation protocols for reliable assessment.
• Analyze model behavior and identify weaknesses for improvement.
• Build tooling and infrastructure for large-scale experimentation and pipelines.
• Establish best practices for assessment, ensuring rigor and reproducibility.
• Collaborate with researchers, engineers, and AI teams to design experiments.
• Contribute to technical reports and client-facing research initiatives.

Skills

Python

C++

Software Engineering

ML / AI Research

Benchmark Design

Evaluation Methodologies

Data Tooling

Communication

Job description

micro1 is seeking a Member of Technical Staff to advance the evaluation and development of frontier coding agents. You will sit at the intersection of AI research, software engineering, and model evaluation, designing benchmarks, methodologies, and data systems that shape how next-generation coding models are measured and improved.

The role emphasizes rigorous experimentation, scalable tooling, and collaboration with researchers and engineers in a remote-first environment.