On-Prem LLM Inference Engineer: GPU & AI Infra

Compunnel, Inc.; Charlotte (NC); On-site; USD 120,000 - 150,000; Full time

Job summary

Compunnel, Inc. in Charlotte, North Carolina is seeking an On-Premise LLM Inference & GPU Systems Engineer. This role involves building, optimizing, and supporting a large-scale enterprise Generative AI infrastructure utilizing NVIDIA H200 GPU clusters and OpenShift AI. Candidates should have significant experience in GPU runtime optimization, Kubernetes orchestration, and managing open-source LLMs.

Key qualifications include 5+ years of experience in relevant roles and hands-on expertise with NVIDIA GPU environments. This position offers a contract opportunity with significant responsibilities across enterprise AI workloads.

Qualifications

• 5+ years as an LLM Systems Engineer or related role.
• Hands-on experience with NVIDIA GPU environments.
• Expertise in deploying and managing inference frameworks.

Responsibilities

• Design and maintain large-scale LLM inference infrastructure.
• Optimize performance of token generation pipelines.
• Manage workload scheduling using Kubernetes.

Skills

NVIDIA GPU environments

Runtime optimization techniques

Token generation pipelines

Kubernetes

OpenShift AI

Tools

vLLM

TensorRT-LLM

RunAI

Job description

Compunnel, Inc. in Charlotte, North Carolina is seeking an On-Premise LLM Inference & GPU Systems Engineer. This role involves building, optimizing, and supporting a large-scale enterprise Generative AI infrastructure utilizing NVIDIA H200 GPU clusters and OpenShift AI. Candidates should have significant experience in GPU runtime optimization, Kubernetes orchestration, and managing open-source LLMs.

Key qualifications include 5+ years of experience in relevant roles and hands-on expertise with NVIDIA GPU environments. This position offers a contract opportunity with significant responsibilities across enterprise AI workloads.