Lead Engineer / Engineer Agentic Platform, AI Platform, xCloud

Home Team Science and Technology Agency (HTX); Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Home Team Science and Technology Agency (HTX) seeks an Engineer/Lead Engineer to join the AI Platform team in Singapore. You will design, build and operate engineering components that enable AI agents to securely reason, use tools, access services and execute multi‑step workflows.

Work includes agent runtimes, tool gateways, secure sandboxes, MCP, workflow persistence and robust CI/CD, with emphasis on security, performance and observability.

Qualifications

• Strong software‑engineering fundamentals.
• Proficiency in at least one language (Python/Go/TypeScript/Java).
• Experience developing APIs, backend services or distributed apps.
• Hands‑on with Docker and Kubernetes.
• Experience with CI/CD and GitLab CI.
• Knowledge of Helm and IaC/GitOps.
• Familiarity with MCP or tool‑integration approaches.
• Understanding of modern LLM concepts and multi‑step model execution.
• Ability to learn new frameworks rapidly.
• Good written and verbal communication.

Responsibilities

• Develop and operate services used to execute AI agents.
• Implement agent execution loops with models, tools, context and app state.
• Integrate agent harnesses and DevSecOps workflows into the platform.
• Define input/output schemas, tool calling and response validation.
• Implement retries, timeouts, cancellation and error handling for model and tool calls.
• Build APIs and SDKs for applications to invoke agent capabilities.
• Secure sandboxed environments and enforce resource limits.
• Ensure observability with Prometheus, Grafana, OpenTelemetry.
• Design MCP servers, clients and gateways for agent communication.
• Implement authentication and authorization for tool calls.

Skills

Python

Go

TypeScript/JavaScript

Java

Docker

Kubernetes

CI/CD

GitLab CI

Helm

IaC / GitOps

LLM concepts

Agentic AI

MCP

Observability

API security

Tools

Prometheus

Grafana

OpenTelemetry

Job description

What the role is:

HTX is Singapore's Science and Technology agency that brings together diverse scientific and engineering capabilities to develop transformative, operationally ready solutions for public safety. As a statutory board under the Ministry of Home Affairs, HTX works at the forefront of science and technology to empower the Home Team with cutting‑edge capabilities. Guided by our mission to amplify, augment and accelerate the Home Team's advantage, we are committed to keeping Singapore the safest place on planet Earth.

xCloud is dedicated to elevating the enterprise experience through cutting‑edge cloud capabilities, including:

• Sustainable enterprise data centres
• Hybrid cloud platforms
• Advanced cloud security measures
• Integrated Development, Security, and Operations (DevSecOps)
• Innovative enterprise Software-as-a-Service (SaaS) solutions
• AI-powered enterprise applications

The jobholder will be part of the AI Platform team, building the common capabilities used by teams across HTX and the Home Team to develop and operate secure agentic AI systems.

You will design, build and operate the engineering components that enable AI agents to securely reason, use tools, access enterprise services and execute multi‑step workflows.

You will work on capabilities including agent runtimes, Model Context Protocol (MCP), tool gateways, skills, secure sandboxes, durable execution, human approvals, evaluations, CI/CD and agent observability.

What you will be working on:

• Develop and maintain services used to execute AI agents.
• Implement agent execution loops integrating models, tools, context and application state.
• Integrate agent harnesses and development frameworks into the platform.
• Implement structured model inputs and outputs, tool calling and response validation.
• Implement retries, timeouts, cancellation and error handling around model and tool calls.
• Develop APIs and SDKs through which applications invoke agent capabilities.
• Implement resource limits governing execution duration, model calls, token consumption and tool calls.
• Troubleshoot complex agent execution failures across application, model and infrastructure layers.
• Develop and operate MCP servers, clients and gateways.
• Implement APIs and connectors that enable agents to securely interact with approved enterprise systems.
• Define machine‑readable tool schemas with clear inputs, outputs and error contracts.
• Implement authentication and authorisation for tool calls. Build tool discovery and registration capabilities.
• Build secure environments for executing model‑generated code and other untrusted agent actions.
• Implement workload isolation using approved container, Kubernetes, microVM or sandbox technologies.
• Apply CPU, memory, storage and execution‑time constraints to sandbox workloads.
• Implement filesystem and network restrictions.
• Build mechanisms for safely injecting temporary and scoped credentials when required.
• Ensure sandbox workloads cannot access platform credentials or unauthorised services.
• Build capabilities for long‑running and multi‑step agent execution.
• Implement workflow state persistence, retry, timeout, scheduling and recovery mechanisms.

What we are looking for:

• 1–5+ years of relevant experience, with appointment and scope as Engineer or Lead Engineer commensurate with technical capability and experience.
• Strong software‑engineering fundamentals.
• Strong proficiency in at least one programming language such as Python, Go, TypeScript/JavaScript or Java.
• Experience developing APIs, backend services or distributed applications.
• Hands‑on experience with Docker and Kubernetes.
• Experience with modern CI/CD practices and tools such as GitLab CI.
• Experience with Helm and Infrastructure as Code or GitOps practices.
• Understanding of modern LLM concepts including model APIs, prompting, structured outputs, tool/function calling and context management.
• Understanding of agentic AI concepts and multi‑step model execution.
• Familiarity with MCP or comparable tool‑integration approaches.
• Experience implementing automated unit and integration testing.
• Knowledge of application and infrastructure observability using technologies such as Prometheus, Grafana or OpenTelemetry.
• Understanding of API security, authentication and authorisation.
• Understanding of DevSecOps and secure software‑development practices.
• Strong troubleshooting ability across software and infrastructure layers.
• Ability to learn new frameworks and technologies rapidly.
• Good written and verbal communication skills.

All new hires are appointed on a two‑year contract in the first instance and will be assessed and considered for permanent tenure over time, based on performance.

As part of the shortlisting process for this role, you may be required to complete a medical declaration and/or undergo further assessment.

All applicants will be updated on the status of their applications within 4 weeks upon closing of the advertisement.

About Home Team Science and Technology Agency (HTX)

HTX is the world’s first Science and Technology agency that integrates a diverse range of scientific and engineering capabilities to innovate and deliver transformative and operationally‑ready solutions for homeland security. As a statutory board of the Ministry of Home Affairs and integral to the Home Team, HTX works at the forefront of science and technology to empower Singapore’s frontline of security. Our shared mission is to amplify, augment and accelerate the Home Team’s advantage and secure Singapore as the safest place on planet earth.