ML Inference Engineer — Terminal Benchmark Architect

Toptal; United States; On-site; USD 138,000 - 276,000; Part time

Job summary

Toptal is seeking ML inference and serving experts to design and deliver expert-level Terminal-Bench tasks. You will turn real, paid ML inference work into hands-on challenges with reproducible environments and working reference solutions.

Tasks are bundled with Docker and Git, and include automated correctness checks. You will ensure tasks reflect real system workloads, including multi-GPU serving, and provide clear documentation of data, rules, and environments so reviewers can validate

Qualifications

• 3+ years doing ML inference or serving hands-on (or a PhD with active research in the subfield)
• Can name the systems, tools, data and rules used in recent work (last 3 years)
• Experience with real-scale model serving (multi-GPU or production)
• Hands-on with Python, CLI, and file-based data formats (CSV/JSON)
• Able to explain how ML results fail and how to detect issues

Responsibilities

• Design expert-level Terminal-Bench tasks from real ML inference work
• Define environments, models, and systems tasks run on in reproducible, solvable ways
• Write reference solutions that run end-to-end in 60–75 minutes
• Build automated checks to prove results are correct and robust
• Package tasks with Docker and Git and document data, tools, and rules
• Iterate tasks based on reviewer feedback to maintain expert-level quality

Skills

ML inference

Python

Shell/CLI

Docker

Git

Model serving systems

Automated tests

Multi-GPU serving

Tools

PyTorch

JAX

vLLM

SGLang

TensorRT-LLM

Job description

Toptal is seeking ML inference and serving experts to design and deliver expert-level Terminal-Bench tasks. You will turn real, paid ML inference work into hands-on challenges with reproducible environments and working reference solutions.

Tasks are bundled with Docker and Git, and include automated correctness checks. You will ensure tasks reflect real system workloads, including multi-GPU serving, and provide clear documentation of data, rules, and environments so reviewers can validate