Lead Engineer/ Engineer, MLOps & AI Inference, AI Platform, xCloud

Home Team Science and Technology Agency (HTX); Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

HTX, Singapore's science and technology agency, seeks an Engineer/Lead Engineer in MLOps & AI Inference to build and operate the AI Platform. You will enable teams to develop, deploy and operate AI systems securely at scale, covering model serving, CI/CD, GPU infra and observability.

You will work across MLOps, LLM inference, GPU infra, model serving, and performance tuning, leveraging Kubernetes, KServe, Helm and CI/CD pipelines to deliver reliable AI capabilities for public safety applications.

Qualifications

• 1–5+ years of relevant experience at Engineer/Lead Engineer level.
• Strong hands-on Linux and Python skills.
• Experience deploying containerised apps with Docker and Kubernetes.
• Experience or knowledge of ML/AI infrastructure and model-serving systems.
• Experience with MLOps or model-serving tech such as KServe, MLflow, Kubeflow or equivalent.

Responsibilities

• Build, maintain and improve shared infrastructure to package, deploy, serve and monitor ML models.
• Develop reusable platform components and deployment patterns for productionising models.
• Build solutions using Kubernetes, Helm, KServe, MLflow and related tools.
• Automate infrastructure using Infrastructure as Code and GitOps.
• Support model lifecycle: registration, versioning, validation, deployment, promotion, rollback and retirement.

Skills

Linux

Python

Docker

Kubernetes

ML infra

CI/CD

GitOps

Prometheus

LLM inference

GPU tooling

Tools

KServe

MLflow

Kubeflow

TensorRT-LLM

Job description

What the role is:

HTX is Singapore's Science and Technology agency that brings together diverse scientific and engineering capabilities to develop transformative, operationally ready solutions for public safety. As a statutory board under the Ministry of Home Affairs, HTX works at the forefront of science and technology to empower the Home Team with cutting-edge capabilities. Guided by our mission to amplify, augment and accelerate the Home Team's advantage, we are committed to keeping Singapore the safest place on planet Earth.xCloud is dedicated to elevating the enterprise experience through cutting-edge cloud capabilities, including:

• Sustainable enterprise data centres
• Hybrid cloud platforms
• Advanced cloud security measures
• Integrated Development, Security, and Operations (DevSecOps)
• Innovative enterprise Software-as-a-Service (SaaS) solutions
• AI-powered enterprise applications

You will be part of the AI Platform team, responsible for building the common AI infrastructure and engineering capabilities that enable teams across HTX and the Home Team to develop, deploy and operate AI systems securely and at scale.As an Engineer / Lead Engineer, MLOps & AI Inference, you will build and operate the platforms that enable machine learning and Generative AI models to run reliably in production. You will work across MLOps, LLM inference, GPU infrastructure, model serving, CI/CD, workload scheduling, performance optimisation and AI observability.

What you will be working on:

• Build, maintain and improve the shared infrastructure used to package, validate, deploy, serve and monitor machine-learning and Generative AI models.
• Develop reusable platform components and deployment patterns that enable AI teams to productionise models consistently.
• Build solutions using Kubernetes, Helm, KServe, MLflow and related cloud-native and MLOps technologies.
• Automate infrastructure and platform configuration using Infrastructure as Code and GitOps practices.
• Support model lifecycle management, including model registration, versioning, validation, deployment, promotion, rollback and retirement.
• Troubleshoot production problems across application, model-serving and infrastructure layers.
• Contribute to improving the reliability, scalability, security and developer experience of the AI Platform.
• Deploy and operate LLM inference services using technologies such as vLLM, SGLang, TensorRT-LLM and equivalent high-performance inference runtimes.
• Configure and optimise inference parameters such as continuous batching, chunked prefill, prefix caching, request scheduling and GPU-memory utilisation.
• Implement and optimise KV-cache management, including cache utilisation, reuse and offloading across GPU memory, host memory and high-performance storage.
• Evaluate and implement model quantisation approaches such as FP8, INT8 and INT4 where appropriate.
• Develop and maintain repeatable benchmarks for evaluating models, inference engines, hardware and configuration changes.
• Develop and operate production model-serving services using KServe, Kubernetes, Helm and approved inference runtimes.
• Implement reusable deployment configurations for AI workloads.
• Configure health checks, readiness checks, resource requirements and scaling policies for model endpoints.
• Deploy and operate GPU workloads on Kubernetes.
• Configure GPU scheduling using Kai Scheduler and other approved Kubernetes scheduling technologies.
• Implement workload queues, resource quotas, priorities, pre-emption and gang scheduling where required.

What we are looking for:

• 1–5+ years of relevant experience, with role scope and appointment as Engineer or Lead Engineer commensurate with experience and demonstrated technical capability.
• Strong hands‑on skills in Linux and Python.
• Experience developing and deploying containerised applications using Docker and Kubernetes.
• Experience or strong technical knowledge of ML/AI infrastructure and model‑serving systems.
• Experience with MLOps or model‑serving technologies such as KServe, MLflow, Kubeflow or equivalent solutions.
• Experience developing and maintaining CI/CD pipelines, for example using GitLab CI.
• Knowledge of Kubernetes deployment and packaging technologies such as Helm.
• Experience with GitOps or Infrastructure as Code practices.
• Understanding of modern LLM inference, including concepts such as:
  • Prefill and decode
  • KV cache
  • Batch
  • Context length
  • GPU memory utilisation
• Experience with or willingness to develop deep expertise in inference technologies such as vLLM, SGLang or TensorRT-LLM.
• Experience monitoring applications or infrastructure using Prometheus, Grafana, OpenTelemetry or equivalent technologies.
• Ability to use benchmark and telemetry data to troubleshoot system‑performance problems.
• Understanding of DevSecOps practices, including container security, vulnerability scanning and secure CI/CD.
• Strong problem‑solving and troubleshooting skills.
• Good written and verbal communication skills.
• Ability to work effectively with engineers, data scientists and technical stakeholders.
• Strong learning mindset and ability to keep pace with rapidly evolving AI infrastructure technologies.

All new hires are appointed on a two-year contract in the first instance and will be assessed and considered for permanent tenure over time, based on performance.

As part of the shortlisting process for this role, you may be required to complete a medical declaration and/or undertake further assessment.

All applicants will be updated on the status of their applications within 4 weeks upon closing of the advertisement.

About Home Team Science and Technology Agency (HTX)

HTX is the world’s first Science and Technology agency that integrates a diverse range of scientific and engineering capabilities to innovate and deliver transformative and operationally‑ready solutions for homeland security. As a statutory board of the Ministry of Home Affairs and integral to the Home Team, HTX works at the forefront of science and technology to empower Singapore’s frontline of security. Our shared mission is to amplify, augment and accelerate the Home Team’s advantage and secure Singapore as the safest place on planet earth.