Senior Machine Learning Engineer

Acceler8 Talent; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Acceler8 Talent is seeking a Member of Technical Staff to join its AI infrastructure team in San Francisco. You will design and build production-grade ML inference and model serving systems, optimizing latency, throughput, and resource utilization across large-scale AI workloads.

You will work with inference runtimes such as vLLM or TensorRT-LLM, tackle memory management and scheduling challenges, and collaborate with compiler, kernel, and distributed systems engineers to push the performance of

Qualifications

• Strong software engineering fundamentals.
• Experience building ML inference or model serving systems in production.
• Deep understanding of system performance and memory behaviour under production workloads.
• Experience with inference runtimes such as vLLM, TensorRT-LLM, or custom serving frameworks.
• Familiarity with batching, scheduling, concurrency, and KV cache management.
• Experience profiling and optimising latency- and throughput-critical systems.
• Strong Python and C++ development experience.

Responsibilities

• Design and build production-grade ML inference and model serving systems
• Optimise latency, throughput, and resource utilisation across large-scale AI workloads
• Develop execution strategies around batching, scheduling, concurrency, and runtime optimisation
• Improve KV cache management, memory efficiency, and model execution behaviour
• Enable new model architectures and inference techniques to run efficiently in production
• Partner closely with compiler, kernel, networking, and distributed systems engineers to drive end-to-end performance
• Help shape the architecture of a next-generation AI inference platform powering production workloads at scale

Skills

Python

C++

ML inference

Model serving

Latency optimisation

Performance profiling

Concurrency

Batching

Scheduling

KV cache

Job description

Member of Technical Staff – ML Systems & Inference

I am seeking a Member of Technical Staff to join one of the most exciting AI infrastructure companies building the next generation of inference systems.

As AI models continue to grow in size and complexity, the challenge is no longer simply adding more GPUs, it's about making diverse hardware work together efficiently. This team is building the infrastructure that intelligently executes AI workloads across heterogeneous compute, delivering significant improvements in performance, efficiency, and scalability for production AI applications.

You'll join a small, highly technical engineering team solving some of the hardest problems in AI systems, working across inference runtimes, scheduling, memory management, and distributed infrastructure.

What You'll Do:

• Design and build production-grade ML inference and model serving systems
• Optimise latency, throughput, and resource utilisation across large-scale AI workloads
• Develop execution strategies around batching, scheduling, concurrency, and runtime optimisation
• Improve KV cache management, memory efficiency, and model execution behaviour
• Enable new model architectures and inference techniques to run efficiently in production
• Partner closely with compiler, kernel, networking, and distributed systems engineers to drive end-to-end performance
• Help shape the architecture of a next-generation AI inference platform powering production workloads at scale

What We're Looking For:

• Strong software engineering fundamentals
• Experience building ML inference or model serving systems in production
• Deep understanding of system performance, memory behaviour, and optimisation under production workloads
• Experience with inference runtimes such as vLLM, TensorRT-LLM, or custom serving frameworks is highly desirable
• Familiarity with batching, scheduling, concurrency, and KV cache management
• Experience profiling and optimising latency- and throughput-critical systems
• Strong Python and C++ development experience
• Comfortable working in a fast-moving, early-stage environment with significant ownership

This is an opportunity to join an exceptionally well-funded AI infrastructure company with a small, world-class engineering team already supporting production deployments for Fortune 500 and AI-native organisations.

You'll work across compiler systems, GPU kernels, distributed scheduling, inference optimisation, and heterogeneous compute, solving difficult engineering challenges that directly impact how modern AI workloads are executed in production.

If you're passionate about high-performance systems, AI infrastructure, and building technology that will define the future of large-scale inference, this is an opportunity to have an outsized impact.