Onsite LLM Inference and GPU Systems Engineer

Delan Associates, Inc; Charlotte (NC); On-site; USD 150,000 - 190,000; Full time

Job summary

Delan Associates, Inc. in Charlotte, NC is seeking an LLM Inference & GPU Systems Consultant to build and sustain large-scale on-prem AI infrastructure. The role focuses on production inference with NVIDIA H200 clusters and OpenShift AI, with no training pipelines.

You will optimize runtime, deploy vLLM and TensorRT-LLM, manage KV cache and prefill/decode, and oversee the Hugging Face deployment lifecycle while ensuring efficient GPU utilization and Kubernetes orchestration.

Qualifications

• 8+ years of hands-on experience in LLM infrastructure and AI runtime engineering.
• 8+ years hands-on experience with NVIDIA H200 clusters and runtime optimization techniques (KV Cache, prefill/decode).
• Proficiency in OpenShift AI and GPU orchestration tools like RunAI.
• Strong experience with modern inference frameworks, specifically vLLM and TensorRT-LLM.
• Proven track record managing the Hugging Face deployment lifecycle.

Responsibilities

• NVIDIA GPU Runtime Optimization: Drive extreme runtime efficiency and optimization for the token generation pipeline. Specifically manage prefill/decode optimization and KV cache management.
• Inference Serving: Deploy and manage inference engines including vLLM and TensorRT-LLM.
• Hardware Utilization: Optimize GPU throughput tuning, batching strategies, and latency optimization. Manage workload orchestration using RunAI and Kubernetes GPU orchestration.
• Model Lifecycle Management: Oversee the complete Hugging Face model lifecycle, including model onboarding, deployment, and retirement.
• Platform Operations: Operate and maintain the OpenShift AI ecosystem as the primary container platform for GenAI workloads.

Education

8+ years LLM Systems Engineer / AI Infrastructure Runtime experience

Tools

NVIDIA H200 clusters

OpenShift AI

RunAI

vLLM

TensorRT-LLM

Hugging Face deployment

Job description

Delan Associates, Inc. in Charlotte, NC is seeking an LLM Inference & GPU Systems Consultant to build and sustain large-scale on-prem AI infrastructure. The role focuses on production inference with NVIDIA H200 clusters and OpenShift AI, with no training pipelines.

You will optimize runtime, deploy vLLM and TensorRT-LLM, manage KV cache and prefill/decode, and oversee the Hugging Face deployment lifecycle while ensuring efficient GPU utilization and Kubernetes orchestration.