Senior Lead, AI Infrastructure & LLM Serving

JPMorgan Chase & Co.; England; On-site; GBP 90,000 - 130,000; Full time

Job summary

JPMorgan Chase & Co. in the United Kingdom seeks a Senior Lead Software Engineer for the AI and Machine Learning Platform team to build and scale AI infrastructure, focusing on reliability, performance, and cost-efficiency of LLM inference platforms.

You will own end-to-end production readiness, implement observability, and drive incident response, capacity planning, and automated remediation across cloud-based and on-prem GPU environments.

Qualifications

• Hands-on production system design, development, testing and stability.
• Advanced Python for building production-grade services and tooling.
• Proficiency with automation and CI/CD methods.
• Experience with cloud platforms and infrastructure-as-code tools.
• Strong understanding of site reliability engineering and incident management.
• Observability across metrics, logs, and traces in production.
• Kubernetes and container orchestration on cloud and on-premise.
• Experience hosting and serving LLMs on cloud and GPU environments.
• Knowledge of LLM reliability, latency, versioning, and safe rollouts.

Responsibilities

• Design, develop, troubleshoot, and deliver secure, production software for AI infrastructure.
• Build backend services and APIs for reliable AI operation in production.
• Operate and scale LLM serving infrastructure with hosting, routing, batching, and caching.
• Deploy and manage models on cloud and on-prem GPU clusters using IaC and CD pipelines.
• Implement observability with dashboards and actionable alerts for LLM workloads.
• Tune GPU capacity, autoscaling, and cost efficiency using optimization techniques.
• Lead reliability engineering for LLM endpoints including capacity planning and incident response.
• Participate in on-call rotations and perform post-incident RCA and follow-ups.
• Identify recurring issues and automate remediation for stability.
• Build multi-agent orchestration systems and promote reusable patterns.
• Drive enterprise AI engineering practices and secure coding standards across the team.

Skills

Python

Cloud infra

Kubernetes

Observability

SRE

Incident mgmt

CI/CD

AI infra

Security best practices

Tools

KServe

Ray Serve

Triton Inference Server

LangChain

LangGraph

Job description

JPMorgan Chase & Co. in the United Kingdom seeks a Senior Lead Software Engineer for the AI and Machine Learning Platform team to build and scale AI infrastructure, focusing on reliability, performance, and cost-efficiency of LLM inference platforms.

You will own end-to-end production readiness, implement observability, and drive incident response, capacity planning, and automated remediation across cloud-based and on-prem GPU environments.