Senior Lead AI Platform Engineer: LLM Serving & Kubernetes

Aumni; Glasgow; On-site; GBP 90,000 - 140,000; Full time

Job summary

J.P. Morgan seeks a Senior Lead Software Engineer to lead the design and delivery of secure, scalable platform capabilities for AI, ML, and data platforms. You will shape tooling, runtime foundations, and production-grade LLM inference on Kubernetes, collaborating across teams to improve reliability and developer experience.

You will mentor engineers, reinforce inclusive, high-accountability practices, and drive standards for automation, observability, and secure coding across engineering groups.

Qualifications

• Hands-on experience building platform standards and tooling to improve adoption.
• Deep, hands-on experience with large language model inference systems in production.
• Experience operating production services on public cloud platforms such as AWS.
• Ability to design, deploy, and troubleshoot cloud infrastructure components used by platform services in AWS.
• Demonstrated Kubernetes expertise across deployments, scaling, networking, and troubleshooting.
• Understanding of GPU memory architecture and performance trade-offs.
• Understanding inference-time quantization and its impact on latency and throughput.
• Ability to produce architecture/design artifacts and translate into secure, scalable implementations.
• Strong knowledge of SDLC practices including CI/CD, resiliency, and security expectations.
• Experience using enterprise AI-assisted software development tools.

Responsibilities

• Lead the design and delivery of platform standards and tooling to simplify adoption and day-to-day use.
• Engineer and operate production large language model inference services using modern serving engines.
• Drive Kubernetes-based deployment patterns, scaling, networking, and troubleshooting for reliable platform operations.
• Optimize inference performance with a deep understanding of GPU memory behavior and bottlenecks.
• Evaluate inference-time quantization trade-offs balancing latency, throughput, cost and quality.
• Implement secure, high-quality production code and automation for resiliency and observability.
• Establish and maintain architecture/design artifacts with automated validation of non-functional requirements.
• Promote enterprise AI-assisted engineering practices to improve code quality and delivery speed.

Skills

LLM inference systems

Kubernetes expert

AWS cloud

CI/CD practices

Secure coding

AI-assisted tooling

GPU memory optimization

Quantization trade-offs

Architecture artifacts

Tools

vLLM

TensorRT-LLM

SGLang

LLM-D

AWS

Job description

J.P. Morgan seeks a Senior Lead Software Engineer to lead the design and delivery of secure, scalable platform capabilities for AI, ML, and data platforms. You will shape tooling, runtime foundations, and production-grade LLM inference on Kubernetes, collaborating across teams to improve reliability and developer experience.

You will mentor engineers, reinforce inclusive, high-accountability practices, and drive standards for automation, observability, and secure coding across engineering groups.