Principal LLM Inference Engineer — Hybrid Lead

Hewlett Packard Enterprise Company; Spring (TX); Hybrid; USD 160,000 - 303,000; Full time

Job summary

Hewlett Packard Enterprise is seeking a Principal Software Engineer to lead the model runtime within the inference platform used to operate large language models on customer-owned hardware. The role focuses on low tail latency and high GPU utilization, with architecture ownership of engine integration, batching, KV cache, and distributed execution, alongside a Kubernetes orchestration layer.

You will mentor engineers and present technical direction to leadership while collaborating with

Qualifications

• Production experience with LLM inference engines and modification of engine internals.
• Deep understanding of inference internals including batching, KV cache, and quantization.
• Expertise in tensor/pipeline parallelism and GPU memory hierarchy.
• Proficient in Kubernetes architecture, operators, CRDs, controllers and scheduling.
• Strong programming in Go and Python; reading C++/CUDA with Nsight tooling.
• Experience debugging multi-tier workloads (RAG, Agents, etc).
• Excellent analytical, debugging and problem-solving abilities.

Responsibilities

• Define and own the technical direction of the LLM serving deployment and orchestration.
• Collaborate with performance teams to optimize time-to-first-token and tail latency.
• Define distributed inferencing strategies across GPU memory and storage.
• Evaluate runtimes, quantization, and serving approaches for adoption.
• Define orchestration layer for model admission, GPU scheduling and autoscaling.
• Mentor engineers, lead architecture reviews, and present to executives.

Skills

LLM inference engines

Inference internals

Tensor parallelism

Kubernetes architectures

Go & Python

Multi-tier debugging

Analytical ability

Education

Degree in Computer Science or related field

Job description

Hewlett Packard Enterprise is seeking a Principal Software Engineer to lead the model runtime within the inference platform used to operate large language models on customer-owned hardware. The role focuses on low tail latency and high GPU utilization, with architecture ownership of engine integration, batching, KV cache, and distributed execution, alongside a Kubernetes orchestration layer.

You will mentor engineers and present technical direction to leadership while collaborating with