Principal AI Inference Engineer (Hybrid)

Hewlett Packard Enterprise; Spring (TX); Hybrid; USD 152,000 - 349,000; Full time

Job summary

Hewlett Packard Enterprise Development LP is seeking a Principal Software Engineer to lead the model runtime for the AI inference platform, focusing on low tail latency and high GPU utilization. You will drive architecture decisions, batching, and distributed execution across environments.

The role requires guiding a team of engineers, coordinating with performance groups, and leveraging Kubernetes to deploy and optimize the stack on-prem and cloud. Hybrid work is available.

Qualifications

• Minimum 12 years of software engineering experience, including +1 year on LLM inference runtimes.
• Degree in Computer Science or related field.
• Strong Go, Python, C++/CUDA development and debugging skills.

Responsibilities

• Define architecture of the model runtime, engine integration and distributed execution.
• Mentor engineers, lead design reviews and present directions to stakeholders.
• Collaborate with performance teams to optimize latency and GPU utilization.

Skills

LLM inference

Go

Python

C++

CUDA

Education

Bachelor's degree in Computer Science

Tools

Kubernetes

Nsight

TensorRT-LLM

Job description

Hewlett Packard Enterprise Development LP is seeking a Principal Software Engineer to lead the model runtime for the AI inference platform, focusing on low tail latency and high GPU utilization. You will drive architecture decisions, batching, and distributed execution across environments.

The role requires guiding a team of engineers, coordinating with performance groups, and leveraging Kubernetes to deploy and optimize the stack on-prem and cloud. Hybrid work is available.