ML Systems Engineer: LLM Inference & Benchmarking

CommonAI CIC; Cambridge; Hybrid; GBP 50,000 - 75,000; Full time

Benefits offered by this job

Pension

Professional development

Networking opportunities

Office near Cambridge train station

Job summary

CommonAI CIC in Cambridge is seeking engineers to build reproducible LLM workloads, benchmark tools, and performance measures. You will characterise workloads, cache effects, and tool interactions, collaborating with measurement and simulator teams to ensure results reflect real-world conditions.

A strong focus on reproducibility and validating AI coding agents is essential. You will join a hybrid work environment with opportunities to influence UK AI infrastructure and policy-relevant research.

Qualifications

• Experience building or evaluating ML systems, inference services or benchmarks.
• Strong software skills for reliable benchmark tools and results analysis.
• Understanding of LLM inference and how serving choices affect performance.
• Care in designing reproducible comparisons and interpreting results.
• Experience using AI coding agents and validating code and data transformations.
• Experience with vLLM, SGLang or similar serving software would be valuable.

Responsibilities

• Build and maintain reproducible LLM serving workloads, configurations and benchmark tools.
• Characterise behaviour such as request arrivals, prompt/output lengths, caching, batching, tool interactions.
• Define performance measures alongside application-quality and service requirements.
• Collaborate with measurement engineers to reproduce workloads on physical systems.
• Work with simulator engineers to develop workload representations that preserve dependencies.
• Investigate how workload changes affect conclusions and keep benchmarks relevant.
• Document datasets, model and software versions, configurations and provenance.

Skills

Experience building ML systems

Benchmark tools development

LLM inference understanding

Reproducible comparisons

AI coding agents experience

Experience with vLLM/SGLang

Job description

CommonAI CIC in Cambridge is seeking engineers to build reproducible LLM workloads, benchmark tools, and performance measures. You will characterise workloads, cache effects, and tool interactions, collaborating with measurement and simulator teams to ensure results reflect real-world conditions.

A strong focus on reproducibility and validating AI coding agents is essential. You will join a hybrid work environment with opportunities to influence UK AI infrastructure and policy-relevant research.