ML Inference Benchmark Engineer – LLM Workloads

CommonAI C.I.C.; Town of Cambridge; Hybrid; AUD 134,000 - 192,000; Full time

Benefits offered by this job

Hybrid Cambridge office

Job summary

CommonAI CIC is seeking engineers to build reproducible LLM serving workloads and benchmarks. You will connect application requirements to physical measurements and our simulation platform, focusing on long-context inference and agentic workloads.

You will work with measurement and simulator engineers to preserve workload behavior while supporting evidence-based infrastructure decisions for investments and compute resources.

Qualifications

• Experience building ML systems and benchmarks.
• Ability to define workload measures and performance metrics.
• Strong software engineering and debugging skills.

Responsibilities

• Build and maintain reproducible LLM serving workloads and benchmarks.
• Characterise request patterns, prompt lengths and caching behaviours.
• Define performance measures aligned with application requirements.
• Reproduce workloads on physical systems with measurement engineers.
• Investigate how workload changes affect conclusions and keep benchmarks relevant.
• Document configurations, provenance, and usage for reproducibility.
• Explore agentic workloads and long-context scenarios with two systems.
• Leverage ML serving tools and validate code and data transformations.

Skills

ML systems

Benchmark tools

LLM inference

Python

Software debugging

Tools

vLLM

SGLang

Job description

CommonAI CIC is seeking engineers to build reproducible LLM serving workloads and benchmarks. You will connect application requirements to physical measurements and our simulation platform, focusing on long-context inference and agentic workloads.

You will work with measurement and simulator engineers to preserve workload behavior while supporting evidence-based infrastructure decisions for investments and compute resources.