Senior LLM Inference Runtime Engineer

Hewlett Packard Enterprise; Fort Collins (CO); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health And Wellbeing Benefits

Personal And Professional Development

Job summary

Hewlett Packard Enterprise seeks an experienced software engineer to design, implement, and operate core components of an enterprise LLM serving runtime. You will drive latency/throughput improvements, contribute to Kubernetes orchestration, and mentor teammates across reviews and technical guidance.

The role requires deep knowledge of inference engines and internals, plus strong Go, Python, C++, and CUDA debugging skills.

Qualifications

• Requires at least eight years of software engineering experience.
• One to two+ years working directly on LLM inference runtimes or production model serving.
• Strong knowledge of inference engines, Kubernetes architecture, Go and Python; debugging and profiling C++/CUDA workloads.

Responsibilities

• Design, implement, and operate major components of an enterprise LLM serving runtime.
• Improve inference latency and throughput.
• Contribute to Kubernetes orchestration, resolve customer issues, and provide technical leadership through reviews and mentoring.

Skills

LLM Inference

Inference Runtime Engineering

vLLM

SGLang

TensorRT-LLM

Continuous Batching

KV Cache Management

Quantization

Speculative Decoding

Tensor And Pipeline Parallelism

NCCL

Kubernetes

Go

Python

C++

CUDA

Education

CS or related degree

Tools

Kubernetes

Go

Python

C++

CUDA

Job description

Hewlett Packard Enterprise seeks an experienced software engineer to design, implement, and operate core components of an enterprise LLM serving runtime. You will drive latency/throughput improvements, contribute to Kubernetes orchestration, and mentor teammates across reviews and technical guidance.

The role requires deep knowledge of inference engines and internals, plus strong Go, Python, C++, and CUDA debugging skills.