Senior Inference SW Engineer

Hewlett Packard Enterprise; Fort Collins (CO); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health And Wellbeing Benefits

Personal And Professional Development

Job summary

Hewlett Packard Enterprise seeks an experienced software engineer to design, implement, and operate core components of an enterprise LLM serving runtime. You will drive latency/throughput improvements, contribute to Kubernetes orchestration, and mentor teammates across reviews and technical guidance.

The role requires deep knowledge of inference engines and internals, plus strong Go, Python, C++, and CUDA debugging skills.

Qualifications

• Requires at least eight years of software engineering experience.
• One to two+ years working directly on LLM inference runtimes or production model serving.
• Strong knowledge of inference engines, Kubernetes architecture, Go and Python; debugging and profiling C++/CUDA workloads.

Responsibilities

• Design, implement, and operate major components of an enterprise LLM serving runtime.
• Improve inference latency and throughput.
• Contribute to Kubernetes orchestration, resolve customer issues, and provide technical leadership through reviews and mentoring.

Skills

LLM Inference

Inference Runtime Engineering

vLLM

SGLang

TensorRT-LLM

Continuous Batching

KV Cache Management

Quantization

Speculative Decoding

Tensor And Pipeline Parallelism

NCCL

Kubernetes

Go

Python

C++

CUDA

Education

CS or related degree

Tools

Kubernetes

Go

Python

C++

CUDA

Job description

Design, implement, and operate major components of an enterprise LLM serving runtime, including engine integration, batching, KV cache management, quantized execution, and distributed GPU execution. Improve inference latency and throughput, contribute to Kubernetes orchestration, resolve customer issues, and provide technical leadership through reviews and mentoring.

Requirements: Requires at least eight years of software engineering experience, including one to two or more years working directly on LLM inference runtimes or production model serving. Candidates should have strong knowledge of inference engines and internals, Kubernetes architecture, Go and Python, and the ability to debug and profile C++/CUDA workloads; a computer science or related degree is specified.

Key Skills: LLM Inference, Inference Runtime Engineering, vLLM, SGLang, TensorRT-LLM, Continuous Batching, KV Cache Management, Quantization, Speculative Decoding, Tensor And Pipeline Parallelism, NCCL, Kubernetes, Go, Python, C++, CUDA

Benefits: Health And Wellbeing Benefits, Personal And Professional Development