Member of Technical Staff: Frontier AI Research (Remote)

Zekai; United States; Remote; USD 240,000 - 350,000; Full time

Benefits offered by this job

Equity compensation

Performance bonuses

Health insurance reimbursement

401(k) with company match

Paid time off

Job summary

Zekai seeks a Member of Technical Staff for Realm Lab to drive frontier AI research through rigorous evaluation and benchmarking. This hands-on role blends ML, data research, and evaluation engineering to build benchmarks, RL environments, and high-quality datasets that advance model capabilities.

You will design experiments, validate evaluations, and develop reproducible pipelines while collaborating with researchers and domain experts to push the boundaries of AI systems.

Qualifications

• Strong technical background in computer science, machine learning, AI, or a related field.
• Hands-on experience designing ML evaluations, benchmark suites, research datasets, or experimental frameworks.
• Proficiency in Python and experience building data pipelines, evaluation tools, or ML research infrastructure.
• Understanding of LLMs, foundation models, post-training techniques, and evaluation methodologies.
• Ability to translate complex real-world workflows into structured tasks, measurable criteria, and research experiments.
• Strong analytical skills in model failure analysis, experimental design, and evaluation quality.
• Ability to work independently and collaborate with researchers, engineers, and domain experts.

Responsibilities

• Own research and evaluation initiatives end-to-end, from task design and dataset development to benchmark validation and performance analysis.
• Design and validate capability evaluations, RL environments, and benchmark suites for computer use, agentic workflows, and emerging model capabilities.
• Develop high-quality datasets, task specifications, evaluation rubrics, and methodologies to improve post-training data quality and model assessment.
• Analyze model behavior through failure analysis, error taxonomies, and capability gap identification to inform research directions.
• Collaborate with researchers and domain experts to design experiments, calibrate evaluation quality, and improve model performance.
• Build reproducible evaluation pipelines and feedback loops to support iterative model development.
• Contribute to benchmarks, research papers, technical reports, and open research artifacts.

Skills

Strong technical background

Hands-on ML evaluations

Python proficiency

LLMs understanding

Experiment design

Analytical skills

Independent work

Job description

Zekai seeks a Member of Technical Staff for Realm Lab to drive frontier AI research through rigorous evaluation and benchmarking. This hands-on role blends ML, data research, and evaluation engineering to build benchmarks, RL environments, and high-quality datasets that advance model capabilities.

You will design experiments, validate evaluations, and develop reproducible pipelines while collaborating with researchers and domain experts to push the boundaries of AI systems.