LLMOps Platform Engineer for Production AI on Kubernetes

Artifintech; Jersey City (NJ); On-site; USD 180,000 - 260,000; Full time

Job summary

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC is seeking an AI Operations Platform Consultant to design, deploy and optimize large-scale GPU-accelerated inference pipelines on Kubernetes, with Triton Inference Server and TensorRT-LLM at the core.

The role covers end-to-end LLMOps, containerized microservices, monitoring, and performance tuning across multi-node clusters in production environments in Jersey City, NJ.

Qualifications

• Experience deploying, managing containerized services at scale on Kubernetes for mission-critical applications.
• Experience deploying, configuring, and tuning LLMs using TensorRT-LLM and Triton Inference Server.
• Setup and operation of AI inference service monitoring for performance and availability.

Responsibilities

• Deploy and manage LLM inference services in production on Kubernetes clusters.
• Optimize performance with model pruning, quantization, and batching.
• Maintain observability with telemetry dashboards for latency, throughput and GPU health.

Skills

GPU-accelerated platforms

Kubernetes

Triton Inference Server

TensorRT-LLM

LLMOps

Tools

OpenShift

Docker

Job description

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC is seeking an AI Operations Platform Consultant to design, deploy and optimize large-scale GPU-accelerated inference pipelines on Kubernetes, with Triton Inference Server and TensorRT-LLM at the core.

The role covers end-to-end LLMOps, containerized microservices, monitoring, and performance tuning across multi-node clusters in production environments in Jersey City, NJ.