GenAI Production Reliability Engineer (L3 Support)

1011 United Overseas Bank Ltd; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

1011 United Overseas Bank Ltd is seeking an experienced SRE/Software Engineer to own L3 incident response for GenAI use cases in production. You will triage, debug, fix defects, and work with vendors to improve reliability across LangChain, Elasticsearch, Redis, and guardrails layers.

Build safe pipelines and enhance observability with Prometheus, Grafana, and OpenTelemetry. The role requires strong Python skills, hands-on LangChain experience, and familiarity with Kubernetes/OpenShift

Qualifications

• 3–4 years in software engineering or SRE with at least 1 year supporting AI or data-intensive services in production.
• Strong Python proficiency.
• Experience reading/fixing LangChain code, FastAPI or Flask services, async patterns and task queues.
• Hands-on LangChain in real projects (chains, tools, retrievers, memory, callbacks).
• Elasticsearch proficiency including Query DSL, relevance tuning, index lifecycle, scaling, Kibana.

Responsibilities

• Own L3 incident response for GenAI use cases in production.
• Triage, debug, fix or coordinate with vendor to resolve defects and improve reliability.
• Review, reproduce and root cause complex bugs across LangChain flows, Elasticsearch, Redis, and Guardrails/LLamaGuard.
• Implement safe changes and hotfixes; build, test, deploy via Jenkins with approvals and rollbacks.
• Drive observability: define signals, add OpenTelemetry traces, and curate Grafana dashboards.
• Coordinate with platform/infra teams during OpenShift, GPUs, vLLM, or NVIDIA Enterprise AI incidents.

Skills

Python

LangChain

FastAPI/Flask

Async patterns

Task queues

Git

Canary/blue-green

Kubernetes/OpenShift

Observability

Incident updates

Education

Bachelor's degree in CS or related

Tools

Elasticsearch

Redis

Kubernetes

OpenShift

Jenkins

Prometheus

Grafana

OpenTelemetry

Job description

1011 United Overseas Bank Ltd is seeking an experienced SRE/Software Engineer to own L3 incident response for GenAI use cases in production. You will triage, debug, fix defects, and work with vendors to improve reliability across LangChain, Elasticsearch, Redis, and guardrails layers.

Build safe pipelines and enhance observability with Prometheus, Grafana, and OpenTelemetry. The role requires strong Python skills, hands-on LangChain experience, and familiarity with Kubernetes/OpenShift