Senior ML Inference Engineer for Terminal Benchmark Tasks

Toptal

Netherlands

Remote

EUR 83,000 - 138,000

Part time

12 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Toptal is building a benchmark to test AI agents on real expert Terminal-Bench tasks. You will design and package expert-level ML inference tasks with end-to-end environments, reference solutions, and automated checks, delivering tasks that take about 60 to 75 minutes of expert effort.

Work is paid per task, with reviewers providing feedback and the opportunity to iterate on difficulty and clarity. Start date is early November 2026; remote work is possible within the Netherlands.

Qualifications

  • 3+ years hands-on ML inference or serving work.
  • Experience with multi-GPU or production serving.
  • Proficient in Python and the command line.
  • Experience with PyTorch or JAX internals and serving stacks.
  • Ability to explain numerical drift and metric bugs.
  • Familiarity with Docker and Git.
  • Available to start by early November 2026.

Responsibilities

  • Design expert-level Terminal-Bench tasks that translate real ML inference work into challenges for AI agents.
  • Define required environments, models, data, and tooling for reproducible tasks.
  • Write reference solutions and verify end-to-end execution in 60–75 minutes.
  • Build automated checks to prove correctness and handle drift or metric issues.
  • Package tasks with Docker and Git and document systems and rules for reviewers.
  • Iterate on tasks based on reviewer feedback and maintain expert-level fairness.

Skills

ML inference
Model serving
PyTorch
JAX
Python
Command line
Data formats

Tools

Docker
Git

Job description

Toptal is building a benchmark to test AI agents on real expert Terminal-Bench tasks. You will design and package expert-level ML inference tasks with end-to-end environments, reference solutions, and automated checks, delivering tasks that take about 60 to 75 minutes of expert effort.

Work is paid per task, with reviewers providing feedback and the opportunity to iterate on difficulty and clarity. Start date is early November 2026; remote work is possible within the Netherlands.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

ML Inference Engineer â Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)
ML Inference Engineer â Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)

Toptal • Netherlands

Remote
EUR 83,000 - 138,000
Science & Clinical AI Benchmarking Lead
Science & Clinical AI Benchmarking Lead

Braintrust • Netherlands

On-site
EUR 50,000 - 62,000
Dutch AI Benchmark Engineer & Multilingual Evaluator
Dutch AI Benchmark Engineer & Multilingual Evaluator

LILT AI • Amsterdam

Remote
EUR 83,000 - 165,000
Flexible schedule
Competitive pay
Remote work possible
Remote Software Engineer - AI Task Designer for Models
Remote Software Engineer - AI Task Designer for Models

Askable • Netherlands

Remote
Confidential
AI Benchmark Engineer
AI Benchmark Engineer

Hamilton Barnes ? • Netherlands

Hybrid
EUR 85,186 - 144,816
ML Engineer (MSc. or PhD)
ML Engineer (MSc. or PhD)

Micro1 • Netherlands

Remote
EUR 75,000 - 150,000
Fully remote
Flexible schedule
Backend Developer - AI Trainer (Netherlands)
Backend Developer - AI Trainer (Netherlands)

Anyone AI Inc. • Netherlands

Remote
EUR 83,000 - 165,000
Staff ML Engineer: Inference & Performance
Staff ML Engineer: Inference & Performance

Nebius Group • Amsterdam

On-site
EUR 120,000 - 190,000
Competitive compensation
Career growth and learning opportunity
Flexibility and ownership
+2
Senior Staff Engineer (Machine Learning) - 45391
Senior Staff Engineer (Machine Learning) - 45391

Turing • Netherlands

Remote
EUR 80,000 - 120,000
Work in a fully remote environment
Opportunity to work on cutting-edge AI projects
Flexible commitments with minimum hours
AI Inference Engineer
AI Inference Engineer

Next Ventures • Amsterdam

Hybrid
EUR 120,000 - 180,000
Hybrid work model