ML Inference & Systems Architect

Acceler8 Talent; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 280,000; Full time

Job summary

Acceler8 Talent in the Bay Area, CA is seeking a Member of Technical Staff - ML Systems & Inference to join onsite. You will help build the orchestration layer for next-generation AI workloads and work across ML systems, inference, and distributed systems teams.

The role focuses on producing production inference systems, optimizing scheduling and memory management, and improving KV cache efficiency to push AI infrastructure forward.

Qualifications

• Strong software engineering fundamentals.
• Experience with ML inference or model serving.
• Knowledge of distributed systems and performance optimization.
• Python and/or C++.
• Experience with vLLM, TensorRT-LLM, CUDA, or similar is a plus.

Responsibilities

• Build production inference systems, optimize scheduling and memory management.
• Improve KV cache efficiency and performance of AI workloads.
• Collaborate with compiler, kernel, and distributed systems engineers to advance AI infrastructure.

Skills

Strong software engineering

ML inference or model serving

Distributed systems and performance

Python

C++

vLLM

TensorRT-LLM

CUDA

Tools

CUDA

TensorRT-LLM

vLLM

Job description

Acceler8 Talent in the Bay Area, CA is seeking a Member of Technical Staff - ML Systems & Inference to join onsite. You will help build the orchestration layer for next-generation AI workloads and work across ML systems, inference, and distributed systems teams.

The role focuses on producing production inference systems, optimizing scheduling and memory management, and improving KV cache efficiency to push AI infrastructure forward.