MTS — AI Systems & Evaluation (Remote)

Zekai; United States; Remote; USD 240,000 - 350,000; Full time

Benefits offered by this job

Equity

Health insurance

Paid time off

401K match

Job summary

Zekai is seeking a Member of Technical Staff (MTS) to operate at the intersection of research, data, and real-world AI systems. This remote, full-time role focuses on improving model and system performance through rigorous evaluation, failure analysis, and iterative development.

You will work with researchers, domain experts, and operators to ensure experimental work yields defensible research signals that translate into meaningful improvements in deployed systems.

Qualifications

• Experience designing ML-oriented datasets, evaluation frameworks, and QA processes.
• Ability to translate messy, real-world system behavior into structured research and evaluation opportunities.
• Comfort operating in ambiguity with ownership and decisive action.

Responsibilities

• Own research and evaluation initiatives end-to-end: problem framing, data design, quality calibration, and signal validation.
• Design ML-oriented data systems, including task definitions, annotation schemas, rubrics, incentives, and pipelines optimized for downstream model performance.
• Analyze model and system failures to identify root causes, edge cases, and opportunities for improvement.
• Translate ambiguous, real-world behavior into structured evaluation frameworks and new data categories.
• Work closely with researchers and domain experts to calibrate quality early and continuously raise the signal bar.
• Iterate rapidly on evaluations, datasets, and feedback loops to improve system performance.
• Act as a quality gate: block claims, pause work, or force scope changes when signal strength or data integrity is insufficient.
• Partner with cross-functional and client-facing teams to translate research progress into clear, credible narratives grounded in evidence.
• Identify gaps in data or evaluation coverage and recommend where to invest, iterate, or stop based on learnings and impact.

Skills

Research signal quality

ML evaluation design

Data systems design

Cross-functional collaboration

Job description

Zekai is seeking a Member of Technical Staff (MTS) to operate at the intersection of research, data, and real-world AI systems. This remote, full-time role focuses on improving model and system performance through rigorous evaluation, failure analysis, and iterative development.

You will work with researchers, domain experts, and operators to ensure experimental work yields defensible research signals that translate into meaningful improvements in deployed systems.