Senior LLM Inference Engineer — Hybrid (Go/Python)

Hewlett Packard Enterprise Development LP; Spring, Northern (TX, KY); Hybrid; USD 144,000 - 273,000; Full time

Job summary

Hewlett Packard Enterprise in Spring, Texas, with hybrid work expectations, seeks a Senior Software Engineer to advance the model runtime for AI Essentials. You will work on engine integration, batching, KV cache reuse, and distributed execution, collaborating across teams to optimize latency and GPU utilization on customer-owned hardware.

The role requires deep knowledge of LLM inference, Kubernetes, and Go/Python proficiency, with opportunities to mentor teammates and contribute to code

Responsibilities

• Design, implement, and own major components of the LLM serving deployment, including engine integration, batching, KV cache management and reuse, and quantized execution.
• Partner with inference engineering teams and contribute to improving time-to-first-token, inter-token latency, throughput per GPU, and P95/P99 tail latency.
• Build and operate distributed execution capabilities, including disaggregated prefill/decode, tensor and pipeline parallelism, and KV cache offload across GPU memory, host memory, and RDMA-attached storage.

Job description

Hewlett Packard Enterprise in Spring, Texas, with hybrid work expectations, seeks a Senior Software Engineer to advance the model runtime for AI Essentials. You will work on engine integration, batching, KV cache reuse, and distributed execution, collaborating across teams to optimize latency and GPU utilization on customer-owned hardware.

The role requires deep knowledge of LLM inference, Kubernetes, and Go/Python proficiency, with opportunities to mentor teammates and contribute to code