AI Production SRE: Secure, Compliant Deployments

LeoTechnologies; Boca Raton (FL); On-site; USD 150,000 - 210,000; Full time

Job summary

LeoTech is hiring a DevOps / SRE to deploy, operate, and harden AI systems that support corrections operations and intelligence analysis. You will own the path from model prototypes to production, in infrastructure meeting CJIS, FedRAMP, or equivalent standards.

You will design deployment platforms for LLM apps, build CI/CD pipelines, and maintain GPU clusters, vector/graph DBs, and Kubernetes-based AI workloads across cloud, on-prem, or air-gapped environments.

Qualifications

• 5+ years in DevOps, SRE, or platform engineering.
• 2+ years operating ML/AI workloads in production.
• Strong fluency with Kubernetes, container orchestration, and infrastructure-as-code (Terraform, Pulumi).
• Solid Python skills for tooling and automation; Bash and at least one systems language is a plus.

Responsibilities

• Design and operate deployment platform for LLM apps and AI workloads across cloud, on-prem, and air-gapped environments.
• Build CI/CD pipelines for model and application delivery with registries, versioning, evaluation gates, and rollback paths.
• Stand up and maintain inference infrastructure: GPU clusters, model serving, vector and graph databases, and related services.
• Operate Kubernetes as the backbone for AI workloads with GPU scheduling, autoscaling, and isolation.
• Implement observability for AI systems focusing on token throughput, model drift, logs, and cost per request.
• Harden environments to CJIS/FedRAMP/StateRAMP/DoD controls with encryption, key management, and audit logging.
• Enforce data segregation and need-to-know access via network policy, IAM, and secret management.
• Build deployment patterns for air-gapped or classified enclaves with offline distribution and signed artifacts.
• Manage incident response: runbooks, on-call rotations, blameless postmortems for AI failures.
• Collaborate with data scientists, security, and compliance to ship safely while maintaining reliability.

Skills

Kubernetes

Terraform

Pulumi

Python

Bash

Security

Observability

GitOps

GPU infra

Tools

EKS

AKS

GKE

Triton

vLLM

Weaviate

Qdrant

Milvus

Neo4j

Neptune

Prometheus

Grafana

OpenTelemetry

Loki

Datadog

MLflow

LangSmith

Langfuse

Weights & Biases

Job description

LeoTech is hiring a DevOps / SRE to deploy, operate, and harden AI systems that support corrections operations and intelligence analysis. You will own the path from model prototypes to production, in infrastructure meeting CJIS, FedRAMP, or equivalent standards.

You will design deployment platforms for LLM apps, build CI/CD pipelines, and maintain GPU clusters, vector/graph DBs, and Kubernetes-based AI workloads across cloud, on-prem, or air-gapped environments.