Remote AI Inference Benchmark Engineer

Silicon Data; United States; Remote; USD 140,000 - 200,000; Full time

Job summary

Silicon Data is seeking an AI Benchmark Engineer to build and run the inference performance side of SiliconMark. You will create automated, containerized benchmarking harnesses across engines and hardware, and own latency, throughput, and per-watt metrics.

This is a hands-on role requiring strong Python, experience with production inference engines (vLLM, TensorRT-LLM, SGLang, TGI), and deep knowledge of batching, KV cache, quantization, and parallelism. Remote work with ET overlap is offered.

Qualifications

• 4+ years of engineering experience in LLM inference, model serving, or performance engineering.
• Strong Python. You write automation and tooling that others can run and trust, not one-off scripts.
• Direct experience with at least one production inference engine (vLLM, TensorRT-LLM, SGLang, TGI, or equivalent).
• Working knowledge of what drives inference performance: batching/scheduling, KV cache, quantization, tensor/pipeline parallelism, memory bandwidth, prefill vs decode.

Responsibilities

• Build and maintain the inference benchmarking harness across engines, models, and hardware targets.
• Own inference metrics: latency, throughput, goodput, and cost per token.
• Design workload profiles that reflect real usage patterns.
• Capture and validate full configuration fingerprints for repeatable results.
• Establish run repeatability and statistical rigor; manage outliers.

Skills

Python

Automation tooling

Inference performance

Benchmarking

Linux

Experimentation

Tools

vLLM

TensorRT-LLM

SGLang

TGI

Job description

Silicon Data is seeking an AI Benchmark Engineer to build and run the inference performance side of SiliconMark. You will create automated, containerized benchmarking harnesses across engines and hardware, and own latency, throughput, and per-watt metrics.

This is a hands-on role requiring strong Python, experience with production inference engines (vLLM, TensorRT-LLM, SGLang, TGI), and deep knowledge of batching, KV cache, quantization, and parallelism. Remote work with ET overlap is offered.