Device Lab and Benchmark Engineer

Desert Ant Labs; Amsterdam; On-site; EUR 90,000 - 120,000; Full time

Job summary

Desert Ant Labs in Amsterdam is seeking a role to build the device lab and benchmarking system used to publish latency, memory, and accuracy figures for our models. You will run benchmarks on the latest hardware and prepare reports that highlight bottlenecks and proposed fixes.

You will translate results into charts, model card sections, and blog posts, and prepare demo apps with go-to-market and marketing teams.

Qualifications

• Know what skews a mobile benchmark: thermal throttling, background load, warm vs cold runs, charging state.
• Have measured models on Core ML, LiteRT, or ONNX Runtime and can explain why results vary between runs.

Responsibilities

• Keep an inventory of all team hardware: laptops, phones, tablets, and training machines.
• Send bottleneck reports to the model and runtime engineers, with a profile and a proposed fix.
• Build demo apps and examples with go-to-market and marketing.
• Turn results into charts, model card sections, and blog posts.
• Run our models on new hardware, from the latest phones to robots.

Skills

Benchmarking knowledge

Hardware lab management

Clean code

Code review

Tools

Core ML

LiteRT

ONNX Runtime

Claude Code

Job description

Every model we ship needs latency, memory, and accuracy numbers from real devices. Build the lab and the tools that produce those numbers.

• Build the device lab and the benchmark system behind every number we publish.
• You’ve built tools engineers use every day, and know what skews a mobile benchmark.
• You live in or near Amsterdam, where the lab is.

Location Amsterdam

Type Full time

About the role

Customers compare our models by the benchmarks we publish. A phone runs a model more slowly when the phone is warm or charging, so the lab controls for temperature, charging, and background load before a number goes public.

You also report where a model runs slowly, with a profile and a proposed fix for the model and runtime engineers. You test our models on hardware we don't support yet, including new phones and robots. You build demo apps with go-to-market and marketing that show a model running on a specific device.

What you will do

• Keep an inventory of all team hardware: laptops, phones, tablets, and training machines.
• Send bottleneck reports to the model and runtime engineers, with a profile and a proposed fix.
• Build demo apps and examples with go-to-market and marketing.
• Turn results into charts, model card sections, and blog posts.
• Run our models on new hardware, from the latest phones to robots.

You might be a fit if you

• Know what skews a mobile benchmark: thermal throttling, background load, warm versus cold runs, and charging state.
• Have measured models on Core ML, LiteRT, or ONNX Runtime and can explain why results vary between runs.
• Live in or near Amsterdam, where the lab is.
• Write clean, tested code, and spot a weak change in review, whether a person or an agent wrote it.
• Plan and run your work through coding agents such as Claude Code, with a low tolerance for slop.

How we work

Start what needs starting without waiting to be asked, and finish what you start. Take on work outside your role when a project needs you.

We ship quickly, so we cut scope until only the part users notice is left. Anyone can comment on your work or redo your draft, and we say early when work isn't ready. We read that feedback as help.

We judge the work by what shipped and what changed because of it. Nobody counts hours.