MLOps & SRE Lead — Dev Experience

HTX (Home Team Science & Technology Agency); Singapore; On-site; SGD 120,000 - 190,000; Full time

Job summary

HTX is Singapore's Science and Technology agency bringing together diverse capabilities to deliver transformative, ready-for-use public safety solutions. As the MLOps/SRE Engineer for HTX's developer experience squad, you will deploy, operate and optimize a production LLM system in our secure infrastructure, ensuring a reliable, cost-efficient AI platform for internal teams.

You will work on LLM deployment, infrastructure management, performance optimization, observability, reliability,

Qualifications

• 4+ years of experience in MLOps, SRE, or DevOps roles, with at least 1 year working with ML/AI systems.
• Hands-on experience deploying and operating LLMs in production (vLLM, TGI, TensorRT-LLM, or similar).
• Strong Kubernetes expertise including operators, StatefulSets, and GPU scheduling.

Responsibilities

• LLM Deployment: Deploy and manage LLM models using GPU infrastructure, optimize throughput, latency, and GPU utilization.
• Infrastructure Management: Provision and maintain vector databases, orchestration services, Redis/queue systems, and API gateways.
• Performance Optimization: Profile and tune LLM inference performance with batching, caching and quantization.
• Observability: Implement monitoring with Prometheus, Grafana, DCGM exporters, and Elastic Stack.
• Reliability Engineering: Define SLOs/SLIs, auto-scaling, failure recovery, and chaos testing.
• Cost Optimization: Monitor GPU usage and reduce token costs and compute spend.
• Security & Compliance: Ensure secure operation, manage secrets, and audit logs.
• Incident Response: Participate in on-call, troubleshoot incidents, perform root-cause analysis.
• Capacity Planning: Forecast GPU needs and scale platforms with adoption.

Skills

MLOps

SRE

DevOps

Python scripting

Bash scripting

Observability

Tools

Kubernetes

Terraform

Helm

Kustomize

Milvus

Weaviate

Qdrant

Pinecone

Prometheus

Grafana

DCGM

Elastic Stack

vLLM

TensorRT-LLM

Kubeflow

MLflow

Job description

HTX is Singapore's Science and Technology agency bringing together diverse capabilities to deliver transformative, ready-for-use public safety solutions. As the MLOps/SRE Engineer for HTX's developer experience squad, you will deploy, operate and optimize a production LLM system in our secure infrastructure, ensuring a reliable, cost-efficient AI platform for internal teams.

You will work on LLM deployment, infrastructure management, performance optimization, observability, reliability,