Senior Lead Software Engineer – LLM Infra & Reliability

Jp Morgan Chase; Glasgow; On-site; GBP 110,000 - 150,000; Full time

Job summary

J.P. Morgan is seeking a Senior Lead Software Engineer to shape reliable AI infrastructure at scale and modernize traditional infrastructure management through AI-driven practices.

You will own the reliability, performance, and cost-efficiency of large language model inference platforms, working with cloud, Kubernetes, and on-prem GPU clusters while leading incident response and post-incident analyses.

Qualifications

• Experience designing and operating production AI infrastructure.
• Strong Python production-grade services development.
• Experience with cloud and IaC tooling.
• Familiar with SRE, incident response, and observability.
• Kubernetes and container orchestration experience.
• Understanding of model hosting and cost optimization.
• Commitment to secure coding and reliability.

Responsibilities

• Design, develop, troubleshoot, and deliver secure, high-quality production software and services for AI infrastructure.
• Build backend services and APIs for reliable AI infrastructure in production.
• Operate and scale LLM serving infrastructure with model hosting, routing, batching, and cache optimization.
• Deploy models on cloud containers and on-prem GPU clusters using IaC and CI/CD pipelines.
• Implement observability across logs, metrics, and traces with dashboards and actionable alerting for LLM and GPU workloads.
• Tune GPU and accelerator capacity, autoscaling, and cost efficiency for inference workloads using optimization techniques.
• Lead reliability engineering for LLM endpoints through capacity planning, load and soak testing, safe rollouts, failover, and incident response.
• Participate in on-call rotations, lead incident triage and mitigation, and produce post-incident root-cause analyses.
• Identify recurring operational issues and automate remediation to improve platform stability.
• Build and maintain multi-agent systems with orchestration, including planning, coordination, tool-calling, state/memory management, and workflow control.
• Drive adoption of enterprise-authorized AI-assisted engineering practices and promote consistent validation standards.

Skills

Python

Kubernetes

CI/CD

Cloud platforms

Observability

Model serving

SRE practices

GPU inference

Security practices

Cost optimization

Tools

KServe

Ray Serve

Triton Inference Server

LangChain

Job description

J.P. Morgan is seeking a Senior Lead Software Engineer to shape reliable AI infrastructure at scale and modernize traditional infrastructure management through AI-driven practices.

You will own the reliability, performance, and cost-efficiency of large language model inference platforms, working with cloud, Kubernetes, and on-prem GPU clusters while leading incident response and post-incident analyses.