Senior Applied ML Engineer, Evals & Data

Cardboard Inc.; Bengaluru; On-site; INR 2,500,000 - 5,000,000; Full time

Benefits offered by this job

Founding-team equity

Job summary

Cardboard Inc. is hiring a Senior Applied ML Engineer to own how we measure and improve our agent's quality end-to-end. You will work with product and engineering to ship improvements rather than hand off reports, and you’ll build evaluation datasets from real product usage.

You should have shipped an LLM or agent system used by real customers, and be strong in TypeScript or Python. Bonus: multimodal AI, video, or creative software.

Qualifications

• Shipped and operated an LLM or agent system used by real customers.
• Strong software engineering in TS or Python.
• Experience building offline and online evaluations.
• Ability to turn vague feedback into measurable metrics.

Responsibilities

• Define what 'good' means for our agent and build trusted evaluation datasets.
• Build offline and online evaluations: automated checks, model graders, human review.
• Study real agent runs to identify failure patterns and improve data and methods.
• Add regression checks and release gates; track quality with latency and cost.
• You've shipped and operated AI/agent systems used by customers; not just a demo.

Skills

Python

TypeScript

Evaluation design

Experiment design

LLM/agent systems

Tools

PyTorch

TensorFlow

Job description

About

We're building the future of storytelling and video editing.

We're a small team that moves fast and builds things we're proud of.

We care obsessively about taste: in design, in product, in every detail.

We're backed by a Tier-1 global fund, YC, and founders of billion dollar companies.

Video is the most powerful way humans tell stories. It always has been. But creating it today is still painfully hard.Fragmented tools, steep learning curves, and workflows that get in the way of the actual creative work. We're buildingCardboard to change that.

Cardboard is an AI-first video editor. Our agent understands a user's request, works with their media, and makes realedits on the timeline. When it gets an edit right, it feels like magic. When it gets one wrong, it costs someone theirafternoon. What separates those two outcomes is measurement.

We have the base of an evaluation system. We're hiring a Senior Applied ML Engineer to build the feedback loop on top ofit: the thing that turns production failures into evaluation cases, cases into a quality bar the team trusts, and thatbar into shipped improvements.

This is a senior individual contributor role, and it is not research-only, prompt-only, or QA. You'll own how we measureand improve the quality of Cardboard's agent, end to end, and you'll work with product and engineering to ship theimprovements rather than hand off a report.

What you'll actually do

Define what "good" means for our agent, and build evaluation datasets we trust out of real product usage.

Build offline and online evaluations: automated checks, model graders, and human review where it's the only honestsignal.

Study real agent runs, find the failure patterns, and close them through better data, better evaluation methods, modelselection, and fine-tuning where it earns its keep.

Add regression checks and release gates, and track quality alongside latency and cost.

You've shipped and operated an LLM or agent system that real customers used, not just a demo.

You're a strong software engineer in TypeScript or Python, and you can work across both.

You've built evaluations, datasets, experiments, or AI quality systems before.

You have strong product judgment. You can take a vague complaint about the agent feeling dumb and turn it intosomething measurable, then move the number.

You don't need a PhD or foundation-model training experience. Evidence that you've built reliable AI products mattersmore to us than credentials or any specific framework.

Bonus: multimodal AI, video, media, or creative software.

Bonus: you know experiment design and statistics well.

What success looks like

Within your first six months:

We have a quality baseline for our main agent workflows that the team actually trusts.

Production failures regularly become new evaluation cases.

Important agent changes pass clear regression checks before release.

We can show measurable improvements in key editing workflows.

What you get

You'd be surrounded by people who are absurdly good at what they do. One started coding at 11 and shipped an app with 6M+ downloads in high school. One got into CS engineering at 14 and has been working on distributed systems for 8+ years. One's an ex-founder who took a company to 1.2M users and $300M+ in transactions. That's the team. We're looking for someone who'll raise the bar on how we measure and improve AI quality. Apart from that you'd get:

Competitive salary and founding-team equity.

Unlimited tokens across every AI model. Use whatever you want, as much as you want.

A healthy budget for AI tools and any peripherals you need to do your best work.

Made it this far? We'd love to hear from you.