Senior Applied Machine Learning Engineer (Eval)

Carrerlift; Bengaluru; On-site; INR 1,900,000 - 3,200,000; Full time

Job summary

Cardboard, Bengaluru-based, is seeking a Senior Applied Machine Learning Engineer to own the quality feedback loop for its agentic video editor. The role focuses on turning production failures into trusted evaluations and measurable improvements.

You will define quality standards, build evaluation datasets, and implement offline/online evaluation systems. Strong ownership and experience with LLMs are essential for success.

Qualifications

• Shipping and operating an LLM or agent system used by real customers.
• Strong software skills in TypeScript or Python and ability to work across both languages.
• Experience building evaluations, datasets, experiments or AI quality systems.

Responsibilities

• Define quality standards for Cardboard's AI agent.
• Build reliable evaluation datasets using data from real product usage.
• Create offline and online evaluation systems using automated checks, model-based graders and human review.
• Analyse real agent runs to identify repeated failure patterns.
• Improve agent quality through better datasets, evaluation approaches, model selection and fine-tuning.
• Build regression checks and release gates for important agent updates.
• Monitor AI quality together with latency and cost.
• Partner with product and engineering teams to ship measurable quality improvements.

Skills

LLM systems

Python

TypeScript

Evaluation design

Datasets

Model fine-tuning

Quality assurance

Production monitoring

Job description

About the role:

• Cardboard is hiring a Senior Applied Machine Learning Engineer focused on evaluations and AI quality.
• The role is based onsite in Bengaluru, India.
• The engineer will own the quality feedback loop for Cardboard's agentic video editor.
• The work involves turning failures observed in production into trusted evaluations and measurable improvements.

What you'll work on:

• Define quality standards for Cardboard's AI agent.
• Build reliable evaluation datasets using data from real product usage.
• Create offline and online evaluation systems using automated checks, model-based graders and human review.
• Analyse real agent runs to identify repeated failure patterns.
• Improve agent quality through better datasets, evaluation approaches, model selection and fine-tuning.
• Build regression checks and release gates for important agent updates.
• Monitor AI quality together with latency and cost.
• Partner with product and engineering teams to ship measurable quality improvements.

What we're looking for:

• Candidates should have experience shipping and operating an LLM or agent system used by real customers.
• Strong software engineering skills in TypeScript or Python are required, with the ability to work across both languages.
• Experience building evaluations, datasets, experiments or AI quality systems is expected.
• Applicants should have strong product judgement and be able to convert vague AI-quality issues into measurable problems.
• Candidates should be comfortable working across data, evaluation methods, model selection and fine-tuning.
• Strong ownership as a senior individual contributor is required.

Good to have:

• Experience with multimodal AI, video, media or creative software is considered a plus.
• Experience with human labeling, model graders or fine-tuning is also beneficial.
• Strong understanding of experiment design and statistics is another bonus qualification.

About Cardboard:

• Cardboard is an AI-first video editor building agentic tools that interpret user requests, work with media and make actual edits on a timeline.
• The company is backed by a Tier-1 global fund, YC and founders of billion-dollar companies.

Interview process:

• The process includes Recruiter Screen, Technical Interview, ML & Evaluation Deep Dive, Product & Engineering Interview, and Final Interview.

Note: ClanX is the recruitment partner helping Cardboard hire for this role.