Remote MTS: AI Agent Evaluation & Experimentation

micro1 Inc.; United States; Remote; USD 200,000 - 250,000; Full time

Benefits offered by this job

Equity compensation

Performance bonuses

Remote-first

Job summary

micro1 is seeking a Member of Technical Staff (MTS) for Cortex Lab to develop and evaluate AI agents and LLM-powered systems in real-world enterprise workflows. This hands-on role emphasizes evaluation, failure analysis, rapid experimentation, and close collaboration with researchers, engineers, domain experts, and customers.

You will build Python-based evaluation pipelines, design benchmarks, and contribute tooling to accelerate experimentation.

Qualifications

• Strong background in applied AI research or research engineering.
• Hands-on experience designing evaluations for AI agents and multi-step workflows.
• Strong Python programming and building evaluation pipelines.
• Experience developing benchmarks, datasets, evaluation metrics, and methodologies.

Responsibilities

• Design and implement evaluation frameworks, benchmarks, and datasets for LLM-powered agents.
• Identify and prioritize model, agent, and workflow failures in real-world production environments.
• Run rapid experiments to validate hypotheses and improve agent reliability.
• Collaborate with enterprise customers and researchers to evaluate, deploy, and optimize AI systems.

Skills

Applied AI

Evaluation design

Python programming

Evaluation pipelines

Benchmarks

Datasets

Experimentation

Collaborations

Tools

Python

ML infrastructure

Job description

micro1 is seeking a Member of Technical Staff (MTS) for Cortex Lab to develop and evaluate AI agents and LLM-powered systems in real-world enterprise workflows. This hands-on role emphasizes evaluation, failure analysis, rapid experimentation, and close collaboration with researchers, engineers, domain experts, and customers.

You will build Python-based evaluation pipelines, design benchmarks, and contribute tooling to accelerate experimentation.