Memory-Optimized LLM Inference Engineer

Acceler8 Talent; Cambridge, Northern (MA, KY); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Acceler8 Talent seeks a Member of Technical Staff, ML Systems for an early-stage AI infrastructure company focused on creating a memory layer for large-scale inference. The role centers on building low-level ML systems and optimizing performance across GPUs and accelerators.

You will work on long-context inference, KV-cache management, and reducing the cost of keeping model state close to compute, while contributing to hardware/software co-design and system-level integration with inference

Qualifications

• Experience in low-level ML systems.
• Experience with GPUs/accelerators and performance optimization.
• Familiarity with inference infrastructure tools.

Responsibilities

• Build and optimize LLM inference systems around distributed and expandable memory.
• Work across KV cache, data movement, GPU kernels and serving performance.
• Help shape the underlying hardware architecture using real workload behaviour.

Skills

Low-level ML systems

GPU performance

Hardware/software co-design

Distributed memory systems

Inference workload analysis

Tools

CUDA

vLLM

SGLang

TensorRT-LLM

LMCache

Job description

Acceler8 Talent seeks a Member of Technical Staff, ML Systems for an early-stage AI infrastructure company focused on creating a memory layer for large-scale inference. The role centers on building low-level ML systems and optimizing performance across GPUs and accelerators.

You will work on long-context inference, KV-cache management, and reducing the cost of keeping model state close to compute, while contributing to hardware/software co-design and system-level integration with inference