Data Quality & Annotation Lead

Nimble Robotics; San Francisco (CA); On-site; USD 100,000 - 120,000; Full time

Benefits offered by this job

Unlimited Flexible Time Off

Health Insurance

Paid Parental Leave

Commuter Benefits

Referral Bonus

401k

Equity

Job summary

Nimble Robotics is seeking a Data Quality & Annotation Lead to own the quality bar for demonstration data used to train our robotic systems. Based in San Francisco, you’ll manage site data collection, write acceptance criteria for tasks, and scale the annotation engine with an overnight remote team to keep data flowing to research on schedule.

The role is hands-on, metrics-driven, and builds the end‑to‑end QA framework from scratch.

Qualifications

• 3+ years in data operations, annotation/labeling operations, or data collection QA for ML systems.
• Experience building quality standards from scratch with rubrics, SOPs and QC workflows.
• Experience managing annotation or review teams, including remote/offshore or vendor teams.
• Ability to deliver direct quality feedback to operators/annotators and drive improvement.

Responsibilities

• Own episode acceptance criteria for every task family and translate needs into auditable standards.
• Take over the annotation and episode-review queue and scale the team and workflows.
• Stand up and manage a remote annotation team with overnight turnaround SLA.
• Design and run a sampling-based QA audit program with inter-rater reliability checks.
• Run operator quality feedback loops with scorecards delivered within 24 hours.
• Instrument quality with metrics and dashboards and partner with analytics on reporting.
• Own the certification bar for new operator onboarding and ensure data collection meets it.
• Drive weekly loops with research and engineering on failure modes and task drift.

Skills

Data operations

Annotation management

Quality assurance

Remote team management

Data fluency

Tools

SQL

BI tools

Job description

About Nimble

Nimble is an AI robotics company building the autonomous supply chain to power fast, efficient and economical commerce. We’re training robot AGI to power a proprietary generalist supply chain superhumanoid, the first robot in the world capable of performing thousands of tasks across the supply chain. We’ve raised over $220M at over $1B valuation and formed a strategic alliance with FedEx to build a national network of autonomous warehouses capable of generating many billions in annual revenue. We are a hardcore and obsessed team of the world’s best engineers and operators. If you are obsessed with your craft, enjoy a high-intensity and fast moving high impact environment, are super high agency in getting hard things done and want to be part of building the world’s most legendary robotics company at the most pivotal moment in history, we want to work with you.

We are on a mission to empower and inspire mankind to accomplish legendary feats by inventing robots that liberate us from the menial. We will accomplish this by training robot AGI to invent and build the Autonomous Supply Chain – everything from the inside of factories and warehouses to your front door – powered by generalist superhumanoids.

Our founding team comes from the AI labs at Stanford and Carnegie Mellon and our board of directors include famed robotics and AI legends including Fei-Fei Li (Chief Scientist of AI at Google and Director of Stanford’s AI Lab), Marc Raibert (founder of Boston Dynamics), and Sebastian Thrun (founder of GoogleX, Waymo; Stanford Professor and considered the father of autonomous vehicles).

Let’s be legendary.

Link:Nimble Closes $106 Million Series C Funding Round, Scales Fully Autonomous Fulfillment with FedEx

Link:FedEx Announces Expansion of FedEx Fulfillment With Nimble Alliance

Why Join Nimble?

At Nimble, we are committed to building legendary products, a legendary team, and a legendary legacy. Join us and become part of an ambitious, humble, and resourceful culture where your work will leave a lasting impact on the future of robotics and commerce.

Nimble’s Core Values:

Be relentlessly resourceful -Challenge conventions and overcome obstacles.

Be legendary -Be the very best and do work that inspires.

Be humble -Prioritize growth, learning, and doing whatever is needed to further the mission.

Be dependable -Take ownership and deliver with high agency.

The Role

Nimble Brain turns real-world operational data into the training sets, evaluations, and feedback loops that make our robots smarter every day. The fuel for that engine is human demonstration data — skilled operators across our sites performing and recording the tasks our superhumanoids learn from — and the quality of every episode we collect sets the ceiling on every model we train.

We’re looking for a Data Quality & Annotation Lead to become the first dedicated owner of that quality bar. Own the rubric, not just run it: you’ll write the acceptance criteria that define what a “good” episode is for every task family, build the audit workflows that enforce them as we scale from ~15 to 100+ operators across four sites this year, and stand up the annotation engine — including a remote annotation team turning around episode review overnight — that keeps labeled, trusted data flowing to research on schedule.

This is a hand‑on, metrics‑driven, build‑from‑scratch role. You’ll be based at our San Francisco HQ and spend heavy time at our collection sites, especially during operator ramps. Success looks like an episode acceptance rate that holds steady while the operator base grows 10x — with rubrics, audits, and dashboards that run like clockwork instead of heroics.

Responsibilities

• Own episode acceptance criteria for every task family — translate research and engineering data needs into operational, auditable standards, and version them as model training needs evolve.
• Take over the annotation and episode-review queue in your first weeks, then build the team and workflows that scale it far beyond yourself.
• Stand up and manage a remote annotation team (likely Philippines-based; direct hires or vendor/BPO) with an overnight turnaround SLA — episodes collected today are reviewed and scored before the next shift starts.
• Design and run a sampling‑based QA audit program with explicit coverage targets, including inter‑rater reliability checks that keep annotators and auditors calibrated.
• Run the operator quality feedback loop: operator‑level quality scorecards delivered to site supervisors within 24 hours. You own the standard and the signal; supervisors own the coaching and people decisions.
• Instrument your function: define the quality metrics (episode acceptance rate, audit coverage, feedback latency, operator quality distribution, annotation throughput) and build the operational dashboards your team runs on, partnering with our analytics function, which independently owns org‑wide reporting.
• Own the certification bar for new operator onboarding — no one collects production data without meeting it — while site teams run the day‑to‑day training reps.
• Drive a standing weekly loop with research and engineering on failure modes, task‑spec drift, and what “good” needs to mean next.

Qualifications

• 3+ years in data operations, annotation/labeling operations, or data collection QA for ML systems — robotics, autonomy, or teleoperation data strongly preferred.
• A track record of building quality standards from scratch — rubrics, SOPs, QC workflows — not just executing against existing ones. Be ready to walk us through one you built.
• Experience managing annotation or review teams, including remote/offshore or vendor/BPO teams, and driving their performance against SLAs.
• Experience delivering direct, frequent quality feedback to operators or annotators — including the hard conversations when someone isn’t meeting the bar.
• Data fluency: able to build and own your own reporting and pressure‑test the numbers — SQL, BI tools, or AI‑assisted, we don’t care how — rather than waiting on someone else.
• Meticulous judgment on edge cases, paired with the pragmatism to ship a v1 rubric this week instead of a perfect one next quarter.
• High agency and comfort with ambiguity in a fast‑paced, high‑growth environment — the org will triple around you this year.
• Able to work in person out of our San Francisco HQ, with regular time at our collection sites.
• Alignment with Nimble’s values: relentlessly resourceful, humble, dependable, and committed to legendary impact.

Additional Requirements

• Familiarity with imitation learning / robot learning data — what makes a demonstration usable for model training.
• Experience scaling annotation or rater teams: pods, shift leads, inter‑rater reliability programs.
• Experience selecting and managing offshore annotation vendors, or standing up direct offshore teams (EOR, timezone‑shifted workflows).
• Multi‑site operations experience.

Compensation

The pay range for this position at the start of employment is expected to be between $100,000 and $120,000/year. Your exact offer may vary depending on multiple individualized factors, including job‑related knowledge, skills, and experience. In addition to cash compensation, this position will also receive generous equity for this position.

Culture:

We embrace challenges and strive to make the impossible possible each day. We’re not in this to do what’s easy or to be mediocre. We want to create something legendary and leave our mark on the world. We’re ambitious, we’re gritty, we’re humble and we’re relentlessly resourceful in pursuit of our goals. If this sounds like you then you might be a great fit!

Nimble Robotics, Inc. is an equal opportunity employer. We make all employment decisions based solely on merit. We provide equal employment opportunity to all applicants and employees without discrimination on the basis of race, color, religion, national origin, ancestry, disability, medical condition, genetic information, marital status, sex, gender, gender identity, gender expression, sexual orientation, age, military or veteran status, or any other characteristic protected by applicable state, federal or local laws.

Nimble’s Benefits

Unlimited Flexible Time Off

Enjoy the time you need to travel, rejuvenate, and connect with friends and family.

Health Insurance

Nimble provides medical, dental, and vision insurance through several premier plans and options to support you and your family.

Paid Parental Leave

Enjoy paid bonding time following a birth.

Commuter Benefits

Take the stress out of commuting with access to fully paid parking spots.

Referral Bonus

Get a cash bonus for any friend or colleagues that you refer to us that we end up hiring.

401k

Contribute towards a 401k for retirement planning.

Equity

Be an owner in Nimble through our equity program