Senior Applied Machine Learning Engineer (Eval) - Bengaluru

CLANS.AI PTE. LTD.; Bengaluru; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Cardboard, an AI-first video editor, is hiring a Senior Applied ML Engineer in Bengaluru to own AI quality for its agentic editor, building evaluation datasets and regression checks to drive measurable improvements.

You will work on offline and online evaluations, model selection and fine-tuning, and collaborate with product and engineering to ship tangible improvements, including handling data quality and evaluation methods.

Qualifications

• Experience shipping and operating an LLM or agent system used by real customers.
• Strong software engineering skills in TypeScript or Python, with ability to work across both.
• Experience building evaluations, datasets, experiments, or AI quality systems.
• Strong product judgment and ability to turn vague AI quality issues into measurable problems.
• Ability to work across data, evaluation methods, model selection, and fine-tuning.
• Strong ownership as a senior individual contributor.
• Bonus: Experience with multimodal AI, video, media, or creative software.
• Bonus: Experience with human labeling, model graders, or fine-tuning.
• Bonus: Strong understanding of experiment design and statistics.

Responsibilities

• Define quality standards for Cardboard’s agent.
• Build trusted evaluation datasets from real product usage.
• Build offline and online evaluations using automated checks, model graders, and human review.
• Analyze real agent runs and identify recurring failure patterns.
• Improve agent quality through better data, evaluation methods, model selection, and fine-tuning.
• Build regression checks and release gates for important agent changes.Track AI quality alongside latency and cost.
• Partner with product and engineering teams to ship measurable improvements.

Skills

LLM systems

Python

TypeScript

AI quality systems

Experiment design

Product judgment

Tools

Python

TypeScript

ML tooling

Job description

Senior Applied ML Engineer to own AI quality for Cardboard’s agentic video editor, building evaluation datasets, offline/online evals, regression checks, and feedback loops that turn production failures into measurable improvements.

Company Details

Cardboard is an AI-first video editor building agentic tools that understand user requests, work with media, and make real edits on the timeline. It is backed by a Tier-1 global fund, YC, and founders of billion-dollar companies. Website: https://www.usecardboard.com

Requirements

• Experience shipping and operating an LLM or agent system used by real customers.
• Strong software engineering skills in TypeScript or Python, with ability to work across both.
• Experience building evaluations, datasets, experiments, or AI quality systems.
• Strong product judgment and ability to turn vague AI quality issues into measurable problems.
• Ability to work across data, evaluation methods, model selection, and fine-tuning.
• Strong ownership as a senior individual contributor.
• Bonus: Experience with multimodal AI, video, media, or creative software.
• Bonus: Experience with human labeling, model graders, or fine-tuning.
• Bonus: Strong understanding of experiment design and statistics.

Responsibilities

• Define quality standards for Cardboard’s agent.
• Build trusted evaluation datasets from real product usage.
• Build offline and online evaluations using automated checks, model graders, and human review.
• Analyze real agent runs and identify recurring failure patterns.
• Improve agent quality through better data, evaluation methods, model selection, and fine-tuning.
• Build regression checks and release gates for important agent changes.Track AI quality alongside latency and cost.
• Partner with product and engineering teams to ship measurable improvements.

Job Details

Bengaluru, India

Interview Process

• Recruiter Screen
• Technical Interview
• ML & Evaluation Deep Dive
• Product & Engineering Interview
• Final Interview

Important Note

ClanX is a recruitment partner, helping Cardboard hire Senior Applied ML Engineer, Evals & Data.