Senior AI Infra & LLM Reliability Engineer

Aumni; Glasgow; On-site; GBP 120,000 - 150,000; Full time

Job summary

J.P. Morgan is seeking a Senior Lead Software Engineer to shape reliable AI production systems. You will build and operate large language model serving infrastructure, with cloud and Kubernetes deployments, deep observability, and cost-aware tuning.

Own reliability, performance, and security across end-to-end LLM endpoints in production. You will lead incident response, capacity planning, on-call rotations, and automation to boost platform stability and developer experience, partnering across

Qualifications

• Hands-on system design, development, testing, and operational stability in production.
• Advanced proficiency in Python for building production-grade services and tooling.
• Proficiency with automation and continuous delivery methods.
• Hands-on experience with cloud infrastructure platforms and infrastructure-as-code tooling for delivery and lifecycle management.
• Strong understanding of site reliability engineering practices, including incident management, root-cause analysis, runbooks, and reliability patterns.
• Practical knowledge of observability and instrumentation across metrics, logs, and traces.
• Hands-on experience with Kubernetes and container-based orchestration platforms, including managed cloud variants.
• Experience hosting and serving large language models on cloud-based infrastructure and local GPU environments.
• Knowledge of large language model reliability and risk considerations, including latency and throughput trade-offs, model versioning, prompt and response logging, and safe rollout patterns.
• Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment with demonstrated ability to evaluate, validate, and refine AI-generated outputs for correctness, performance, and security.
• Understanding of responsible AI use in engineering workflows, including data sensitivity considerations and secure handling of inputs/outputs.

Responsibilities

• Design, develop, troubleshoot, and deliver secure, high-quality production software and services for AI infrastructure
• Build backend services and APIs that enable reliable operation of AI infrastructure in production environments
• Operate and scale large language model serving infrastructure, including model hosting, request routing, continuous batching, and cache optimization
• Deploy, host, and lifecycle-manage open-source and proprietary large language models on cloud-based container orchestration platforms and on-premises GPU clusters using reproducible infrastructure as code and continuous delivery pipelines
• Implement observability across logs, metrics, and traces with dashboards and actionable alerting for large language model and GPU workloads
• Tune GPU and accelerator capacity, autoscaling, and cost efficiency for large language model inference workloads using performance optimization techniques such as quantization, parallelism, and speculative decoding
• Lead reliability engineering for large language model endpoints through capacity planning, load and soak testing, safe rollouts, failover, and incident response for outages and model-quality regressions
• Participate in on-call rotations, lead incident triage and mitigation, and produce clear post-incident root-cause analyses and follow-up actions
• Identify recurring operational issues and automate remediation to improve platform stability and developer experience
• Build and maintain multi-agent systems with strong orchestration, including planning, coordination, tool-calling, state and memory management, and workflow control where appropriate
• Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes, while establishing consistent validation standards and promoting reuse of patterns across the team.

Skills

System design

Python

Automation

Cloud infrastructure

Site reliability engineering

Observability

Kubernetes

LLM hosting

LLM reliability

AI-assisted tools

Responsible AI

Tools

KServe

Ray Serve

NVIDIA Triton

Job description

J.P. Morgan is seeking a Senior Lead Software Engineer to shape reliable AI production systems. You will build and operate large language model serving infrastructure, with cloud and Kubernetes deployments, deep observability, and cost-aware tuning.

Own reliability, performance, and security across end-to-end LLM endpoints in production. You will lead incident response, capacity planning, on-call rotations, and automation to boost platform stability and developer experience, partnering across