Senior Applied ML Engineer, Evals & Data

Engg; Bengaluru; On-site; INR 4,000,000 - 6,000,000; Full time

Benefits offered by this job

Founding-team equity

Unlimited AI model tokens

Budget for AI tools

Job summary

Engg in Bengaluru, India, is seeking a Senior Applied ML Engineer to own how we measure and improve Cardboard’s agent end-to-end. This role emphasizes building reliable evaluation systems and turning production signals into a measurable quality bar.

You will define what “good” means, create trusted evaluation data, and collaborate with product and engineering to ship improvements that reduce failures and improve user experience.

Qualifications

• You’ve shipped and operated an LLM or agent system used by real customers.
• You’re a strong software engineer in TypeScript or Python.
• You’ve built evaluations, datasets, experiments, or AI quality systems.
• You have strong product judgment and can turn feedback into measurable improvements.
• You have bonus: multimodal AI, video, media, or creative software.

Responsibilities

• Define what 'good' means for our agent and build trusted evaluation datasets.
• Build offline and online evaluations: automated checks, model graders, and human review.
• Study real agent runs, identify failure patterns, and close gaps through data and model tweaks.
• Add regression checks and release gates, tracking quality alongside latency and cost.

Skills

LLM experience

Python

TypeScript

Evaluations & data

Experiment design

Tools

Python

TypeScript

Job description

Senior Applied ML Engineer, Evals & Data

Bengaluru, India • Full-time

We’re building the future of storytelling and video editing.

We’re a small team that moves fast and builds things we’re proud of.

We care obsessively about taste: in design, in product, in every detail.

We’re solving these hard problems.

We’re backed by a Tier-1 global fund, YC, and founders of billion dollar companies.

AI Engineering Video is the most powerful way humans tell stories. It always has been. But creating it today is still painfully hard. Fragmented tools, steep learning curves, and workflows that get in the way of the actual creative work.

We’re building Cardboard to change that. Cardboard is an AI‑first video editor. Our agent understands a user’s request, works with their media, and makes real edits on the timeline. When it gets an edit right, it feels like magic. When it gets one wrong, it costs someone their afternoon. What separates those two outcomes is measurement. We have the base of an evaluation system. We’re hiring a Senior Applied ML Engineer to build the feedback loop on top of it: the thing that turns production failures into evaluation cases, cases into a quality bar the team trusts, and that bar into shipped improvements. This is a senior individual contributor role, and it is not research‑only, prompt‑only, or QA. You’ll own how we measure and improve the quality of Cardboard’s agent, end to end, and you’ll work with product and engineering to ship the improvements rather than hand off a report.

Responsibilities

• Define what "good" means for our agent, and build evaluation datasets we trust out of real product usage.
• Build offline and online evaluations: automated checks, model graders, and human review where it’s the only honest signal.
• Study real agent runs, find the failure patterns, and close them through better data, better evaluation methods, model selection, and fine‑tuning where it earns its keep.
• Add regression checks and release gates, and track quality alongside latency and cost.

Who you are

• You’ve shipped and operated an LLM or agent system that real customers used, not just a demo.
• You’re a strong software engineer in TypeScript or Python, and you can work across both.
• You’ve built evaluations, datasets, experiments, or AI quality systems before.
• You have strong product judgment. You can take a vague complaint about the agent feeling dumb and turn it into something measurable, then move the number.
• You don’t need a PhD or foundation‑model training experience. Evidence that you’ve built reliable AI products matters more to us than credentials or any specific framework.
• Bonus: multimodal AI, video, media, or creative software.
• Bonus: human labeling, model graders, or fine‑tuning.
• Bonus: you know experiment design and statistics well.

What success looks like

• We have a quality baseline for our main agent workflows that the team actually trusts.
• Production failures regularly become new evaluation cases.
• Important agent changes pass clear regression checks before release.
• We can show measurable improvements in key editing workflows.

What you get

You’ll be surrounded by people who are absurdly good at what they do. One started coding at 11 and shipped an app with 6M+ downloads in high school. One got into CS engineering at 14 and has been working on distributed systems for 8+ years. One’s an ex‑founder who took a company to 1.2M users and $300M+ in transactions. That’s the team. We’re looking for someone who’ll raise the bar on how we measure and improve AI quality.

• Competitive salary and founding‑team equity.
• Unlimited tokens across every AI model. Use whatever you want, as much as you want.
• A healthy budget for AI tools and any peripherals you need to do your best work.