Senior ML Ops Engineer — Remote GPU Inference Platform

Openkrill; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Pragmatike is seeking a hands-on ML Ops Engineer for a fast-scaling distributed cloud infrastructure startup. The role focuses on building scalable, reliable model serving platforms powering real-time AI applications.

You will design and operate core infrastructure for serving ML models at scale, collaborating with infra, platform, and AI teams to ensure high availability, low latency, and cost-efficient inference in a remote-first environment.

Qualifications

• 4+ years in ML Ops, Platform/SRE or similar infrastructure roles focused on ML systems.
• Hands-on experience with model serving frameworks such as vLLM, TGI, Triton, or equivalent.
• Strong background in container orchestration and operating GPU-based workloads in production.
• Experience with MLOps tooling including model registries, experiment tracking, and automated deployment pipelines.
• Proficiency in Python and infrastructure-as-code tools (e.g., Terraform, Helm).
• Understanding of distributed systems, performance tuning, and production reliability engineering.

Responsibilities

• Build and operate production-grade model serving infrastructure using frameworks such as vLLM, TGI, Triton, or equivalent.
• Design and implement robust deployment pipelines with blue/green and canary rollout strategies for ML models.
• Develop and maintain auto-scaling systems, multi-model serving architectures, and intelligent request routing layers.
• Optimize GPU utilization, memory efficiency, network throughput, and model artifact storage performance.
• Design observability systems for tracking inference latency, throughput, GPU usage, cost metrics, and system health.
• Manage model registries and CI/CD pipelines enabling automated and reproducible model deployments.
• Own the full lifecycle of ML systems from development through production, including operational support and on‑call responsibilities.
• Define engineering best practices and contribute to platform scalability in a fast-moving startup environment.

Skills

ML Ops

Python

Kubernetes

CI/CD pipelines

Tools

vLLM

TGI

Triton

Job description

Pragmatike is seeking a hands-on ML Ops Engineer for a fast-scaling distributed cloud infrastructure startup. The role focuses on building scalable, reliable model serving platforms powering real-time AI applications.

You will design and operate core infrastructure for serving ML models at scale, collaborating with infra, platform, and AI teams to ensure high availability, low latency, and cost-efficient inference in a remote-first environment.