LLM Inference Runtime Architect | Performance & Scalability

Hewlett Packard Enterprise; Durham (NC); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health & Wellbeing

Personal & Professional Development

Job summary

Hewlett Packard Enterprise seeks a senior architect to define and own the technical architecture for LLM inference runtimes, focusing on execution efficiency and distributed inferencing strategies on customer-owned hardware. You will lead design reviews and mentor engineers.

You will collaborate with performance teams to optimize latency and throughput, apply continuous batching, and advance GPU memory management and inference internals in a fast-paced AI systems environment.

Qualifications

• Requires at least 12 years of software engineering experience with a minimum of 1 year working directly on LLM inference runtimes.
• Candidates must possess expert-level proficiency in Kubernetes, Python, Go, and deep knowledge of GPU memory hierarchies and inference internals.

Responsibilities

• Define and own the technical architecture for LLM inference runtimes.
• Lead design reviews, mentor engineers, and partner with performance teams to optimize latency and throughput on customer-owned hardware.

Skills

LLM Inference

Kubernetes

Python

Go

C++

CUDA

TensorRT-LLM

vLLM

Distributed Systems

GPU Optimization

Continuous Batching

KV Cache Management

Architecture Design

Performance Engineering

Machine Learning

Job description

Hewlett Packard Enterprise seeks a senior architect to define and own the technical architecture for LLM inference runtimes, focusing on execution efficiency and distributed inferencing strategies on customer-owned hardware. You will lead design reviews and mentor engineers.

You will collaborate with performance teams to optimize latency and throughput, apply continuous batching, and advance GPU memory management and inference internals in a fast-paced AI systems environment.