On-Prem LLM Inference & GPU Systems Architect

NTT DATA North America; Charlotte (NC); On-site; USD 120,000 - 150,000; Full time

Job summary

NTT DATA North America is seeking an On-Premise LLM Inference & GPU Systems Engineer in Charlotte, North Carolina. The role focuses on building and maintaining large-scale on-prem LLM infrastructure using NVIDIA H200 GPU clusters.

Responsibilities include optimizing runtime efficiencies, managing inference serving, and overseeing the complete Hugging Face model lifecycle within the OpenShift AI ecosystem. Ideal candidates should have extensive experience with NVIDIA clusters and modern inference frameworks.

Qualifications

• 5+ years expertise as an LLM Systems Engineer or AI Infrastructure Runtime Engineer.
• 5+ years hands-on experience with NVIDIA H200 clusters and runtime optimization techniques.
• 3+ years experience in OpenShift AI and GPU orchestration tools.

Responsibilities

• Drive extreme runtime efficiency and optimization for the token generation pipeline.
• Deploy and manage inference engines including vLLM and TensorRT-LLM.
• Optimize GPU throughput tuning, batching strategies, and latency optimization.

Skills

NVIDIA H200 clusters

Inference frameworks (vLLM, TensorRT-LLM)

OpenShift AI

GPU orchestration techniques

Tools

Kubernetes

RunAI

Job description

NTT DATA North America is seeking an On-Premise LLM Inference & GPU Systems Engineer in Charlotte, North Carolina. The role focuses on building and maintaining large-scale on-prem LLM infrastructure using NVIDIA H200 GPU clusters.

Responsibilities include optimizing runtime efficiencies, managing inference serving, and overseeing the complete Hugging Face model lifecycle within the OpenShift AI ecosystem. Ideal candidates should have extensive experience with NVIDIA clusters and modern inference frameworks.