Senior AI Infra Engineer - Kubernetes and LLM Serving Remote

SmartRecruiters, Inc.; Northern (KY); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive compensation

Remote-first team

Conferences & training

Hackathons & tech talks

Job summary

Mirantis is building a new enterprise AI infrastructure product to run and govern large language models on customer Kubernetes clusters. You will join a small senior team with broad ownership of the model-serving layer and its path to production.

You will design and build LLM serving infra on Kubernetes, including deployment, GPU scheduling, scaling, and lifecycle management, while packaging for enterprise readiness and ensuring governance and observability.

Qualifications

• 5+ years of software engineering experience in infrastructure, platform, or distributed systems.
• Deep hands-on Kubernetes experience: production workloads and Helm charts.
• Experience with GPU workloads or LLM inference, or strong adjacent systems experience.

Responsibilities

• Design and build LLM serving infrastructure on Kubernetes: deployment, GPU scheduling, scaling, and model lifecycle management.
• Package the platform for enterprise environments: Helm-based installs, upgrades, and restricted/offline networks.
• Integrate the serving layer with the platform's API gateway, identity, and metering services.
• Build the observability for operating GPU inference in production (serving metrics, GPU telemetry).
• Contribute across a multi-service codebase and help set engineering direction through design docs and reviews.

Skills

Go programming

Distributed systems

CI/CD practices

Autonomy

Problem solving

Tools

Kubernetes

Helm charts

CI/CD pipelines

GPU workloads

Open-source contributions

Job description

Mirantis is building a new enterprise AI infrastructure product to run and govern large language models on customer Kubernetes clusters. You will join a small senior team with broad ownership of the model-serving layer and its path to production.

You will design and build LLM serving infra on Kubernetes, including deployment, GPU scheduling, scaling, and lifecycle management, while packaging for enterprise readiness and ensuring governance and observability.