ML Engineer

RustLabs; Connecticut; On-site; USD 120,000 - 165,000; Full time

Job summary

RustLabs is seeking an ML engineer to own core data pipelines and evaluation systems that turn raw tasker output into model-ready training data. This hands-on IC role involves designing annotation schemas with customers, building scalable tooling, and delivering data-quality at scale for frontier AI research.

You will work closely with research teams, ship production ML systems, and contribute to research artifacts.

Qualifications

• 2+ years of professional experience as an ML engineer, research engineer, or similar role.
• Shipped ML systems to production and understand data quality in practice.
• Familiarity with RLHF, SFT, or evaluation methodologies is a strong plus.
• Ability to write clean code, move fast, and talk directly to customers.
• Bonus: open-source contributions or prior AI lab experience.

Responsibilities

• Design annotation pipelines (RLHF, SFT, eval, red-teaming) end-to-end, including schema design and tasker UX.
• Build evaluation infrastructure: automated checks, LLM-as-judge systems, calibration tooling, and inter‑annotator metrics.
• Collaborate with research teams at AI labs to translate data needs into deliverable annotation products.
• Develop internal tooling to enable a distributed tasker workforce to produce high-quality data at scale.
• Contribute to research artifacts (datasets, evals, papers) when relevant work ships.

Skills

Python

ML engineer

Data quality

Customer communication

Production ML

Tools

PyTorch

HuggingFace

Job description

About RustLabs

We’re building the data layer for frontier AI. RustLabs is a high-throughput annotation and evaluation platform used by AI labs to produce training data, RLHF preference signals, and expert evaluations across text, image, code, and multimodal domains. We’re early, well-funded, and working directly with research teams at top labs.

The role

You’ll own the technical core of the platform — the pipelines that turn raw tasker output into clean, model-ready training data. This is a hands‑on IC role with significant ownership: you’ll design annotation schemas with customers, build evaluation infrastructure, write the tooling that ensures data quality at scale, and sit at the intersection of ML research and operations.

What you’ll do

• Design and ship annotation pipelines (RLHF, SFT, eval, red‑teaming) end-to‑end — schema design, tasker UX, quality controls, aggregation, delivery to customers.
• Build evaluation infrastructure: automated checks, LLM-as-judge systems, calibration tooling, and inter‑annotator agreement metrics.
• Work directly with research teams at AI labs to understand their data needs and translate them into shippable annotation products.
• Build the internal tooling that lets a distributed tasker workforce produce gold‑standard data at scale — onboarding flows, qualification tests, payout logic, quality dashboards.
• Contribute to research artifacts (datasets, evals, papers) when relevant work ships.

What we’re looking for

• 2+ years of professional experience as an ML engineer, research engineer, or similar role. Strong Python, comfort with modern ML frameworks (PyTorch, HuggingFace).
• You’ve shipped ML systems to production — you’ve debugged training pipelines, you know what “data quality” actually means in practice, you understand the failure modes of LLM evaluation.
• Familiarity with RLHF, SFT, or evaluation methodologies is a strong plus.
• You can write clean code, move fast, and talk directly to customers.
• Bonus: prior experience at an AI lab, data platform, or annotation company. Bonus: open-source contributions.