Remote Lead, Coding-Agent Evaluation & Benchmarking

Zekai; United States; Remote; USD 200,000 - 260,000; Full time

Benefits offered by this job

Equity compensation

Performance-based bonuses

Health-insurance premium reimbursement

401(k) plan with company match

Paid time off

Job summary

Zekai is seeking a Member of Technical Staff to advance the evaluation and development of frontier coding agents. Remote role open to worldwide applicants.

You will design benchmarks, data systems, and methodologies shaping how next-generation coding models are measured and improved. You will lead end-to-end research, develop datasets and evaluation protocols, and collaborate with researchers and engineers to produce actionable insights and robust assessment pipelines for enterprise-ready AI

Qualifications

• Strong software engineering background with a focus on evaluation
• Experience designing benchmarks, tasks, and rubrics for AI systems
• Proven ability to analyze model behavior and derive actionable improvements
• Ability to build tooling and automate experimentation workflows

Responsibilities

• Design evaluation frameworks for coding agents with benchmarks and scoring
• Lead end-to-end research on measuring coding model performance across tasks
• Develop datasets, golden examples, and evaluation protocols
• Analyze model behavior and identify weaknesses to drive improvements
• Build infrastructure for large-scale experimentation and evaluation pipelines
• Establish best practices for assessment with rigor and reproducibility
• Collaborate with researchers, engineers, and applied AI teams on experiments
• Contribute to technical reports, benchmark studies, and client-facing research

Skills

Analytical thinking

Excellent communication

Research-oriented mindset

Experience with ML/AI evaluation

Tools

Python

C++

Job description

Zekai is seeking a Member of Technical Staff to advance the evaluation and development of frontier coding agents. Remote role open to worldwide applicants.

You will design benchmarks, data systems, and methodologies shaping how next-generation coding models are measured and improved. You will lead end-to-end research, develop datasets and evaluation protocols, and collaborate with researchers and engineers to produce actionable insights and robust assessment pipelines for enterprise-ready AI