Lead Software Engineer - Python / Go & AI/ML

JP Morgan Chase; Glasgow; On-site; GBP 62,000 - 102,000; Full time

Job summary

JPMorganChase is seeking a Lead Software Engineer to advance LLM inference performance, optimization strategy, benchmarking, and efficiency at scale within the AI/ML Data Platform team. You will work with senior engineers and leadership to shape how our platform evolves in a global financial institution environment, focusing on secure, scalable AI deployment and high-impact delivery.

The role offers exposure to state-of-the-art GPU and cloud infrastructure practices, including AWS and

Qualifications

• Formal training or certification on software engineering concepts and advanced experience, preferably in Go or Python.
• Hands-on experience with LLM inference systems such as vLLM, TensorRT-LLM, SGLang, LLM-D, or equivalent production serving engines.
• Strong understanding of GPU memory architecture, including KV cache sizing and dynamics, memory-bandwidth vs compute bottlenecks, and quantization implications at inference time.
• Experience with quantization techniques and real-world tradeoffs at scale.
• Familiarity with speculative decoding and acceptance rate drivers in production workloads.
• Rigorous benchmarking skills using GuideLLM or equivalent tooling with data-backed claims.
• Experience operating in cloud GPU infrastructure at scale (AWS) and Kubernetes-based managed inference services.
• Ability to communicate technical trade-offs to engineering peers and senior stakeholders.
• Hands-on experience using enterprise AI-assisted software development tools and validating AI outputs.
• Understanding of responsible AI in engineering workflows, including data sensitivity, secure handling, and resiliency/security expectations.
• Preferred: experience with disaggregated prefill/decode serving architectures.
• Preferred: familiarity with GPU diagnostics tools such as DCGM, NVML, or XID event tracking.
• Preferred: ML observability and production monitoring for inference workloads.
• Preferred: awareness of LLM inference landscape with benchmarks guiding platform improvements.

Responsibilities

• Execute systematic benchmarking and performance characterization across production LLM workloads.
• Design and run quantization experiments (FP8, INT8/INT4) and measure accuracy delta, throughput, memory reduction, and cost-per-token impact.
• Support speculative decoding strategies across model portfolio and provide configuration recommendations.
• Build GPU efficiency metrics and provide data-driven views of platform efficiency to engineering teams.
• Benchmark platform against external providers and published industry numbers.
• Participate in inference engine upgrade evaluations (schedulers, async tensor parallelism, disaggregated prefill/decode).
• Contribute to GPU chaos engineering efforts including failure scenarios and hardware diagnostics monitoring.
• Leverage enterprise AI coding assist tools while validating outputs through peer review and secure coding standards.
• Apply SDLC tooling knowledge to improve automation and value realization.

Skills

Go

Python

Benchmarking

Tools

vLLM

TensorRT-LLM

SGLang

LLM-D

AWS

Kubernetes

DCGM

NVML

XID

GuideLLM

Job description

Salary: £62,000 - 102,000 per year

Requirements:

• Formal training or certification on software engineering concepts and advanced applied experience, preferably in Go or Python
• Hands-on experience with LLM inference systems such as vLLM, TensorRT-LLM, SGLang, LLM-D, or equivalent production serving engines
• Strong understanding of GPU memory architecture, including KV cache sizing and dynamics, memory-bandwidth versus compute bottlenecks, and the practical implications of quantization at inference time
• Experience with quantization techniques and their real-world tradeoffs at scale
• Familiarity with speculative decoding and the variables that drive acceptance rates in production workloads
• Rigorous benchmarking skills using GuideLLM, custom harnesses, or equivalent tooling, with the ability to support every performance claim with data
• Experience operating in cloud GPU infrastructure at scale, including AWS and Kubernetes-based managed inference services
• Ability to communicate technical trade-offs clearly to engineering peers and senior stakeholders
• Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment, with demonstrated ability to critically evaluate and validate AI-generated outputs
• Understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs and outputs, and adherence to resiliency and security expectations
• Preferred: Experience with disaggregated prefill/decode serving architectures
• Preferred: Familiarity with GPU hardware diagnostics tools such as DCGM, NVML, or XID event tracking
• Preferred: Experience with ML observability and production monitoring for inference workloads
• Preferred: Awareness of the LLM inference competitive landscape with a track record of applying industry benchmarks to drive platform improvements

Responsibilities:

• Execute systematic benchmarking and performance characterization across production LLM workloads, establishing reproducible baselines, identifying regressions, and quantifying the impact of configuration changes before they reach production
• Design and run quantization experiments, including FP8, INT8/INT4 (GPTQ/AWQ), and next-generation precision formats, measuring accuracy delta, throughput improvement, memory reduction, and cost-per-token impact
• Support speculative decoding strategy across the model portfolio, including draft model, n-gram, and multi-token prediction approaches, contributing to acceptance rate measurement and per-workload configuration recommendations
• Build and maintain GPU efficiency metrics covering utilization, memory headroom, cost per 1K tokens, and waste identification, providing engineering teams with a data-driven view of platform efficiency
• Benchmark the platform against external providers and published industry numbers, identifying gaps and contributing to improvement initiatives
• Participate in inference engine upgrade evaluations, including new scheduler architectures, async tensor parallelism, disaggregated prefill/decode, and advanced speculative decoding, supporting systematic validation before production promotion
• Contribute to GPU chaos engineering efforts, including induced failure scenarios, hardware diagnostic monitoring, and detection and recovery measurement
• Leverage enterprise-authorized AI coding assist tools within the work environment to improve code quality, delivery speed, and productivity, while validating outputs through peer review, automated testing, and secure coding standards
• Apply knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation

Technologies:

• AI
• AWS
• Cloud
• Hardware
• Support
• Kubernetes
• LLM
• Marketing
• Python
• Security
• vLLM
• Machine Learning

More:

At JPMorganChase, we are building the infrastructure that powers the next generation of enterprise AI, and this Lead Software Engineer role sits within our AI/ML Data Platform team. We are focused on LLM inference performance, optimization strategy, benchmarking, and efficiency at scale, and this high-impact individual contributor position works closely with senior engineers and engineering leadership to shape how our platform evolves. We offer the opportunity to contribute directly to how one of the worlds largest financial institutions deploys and optimizes AI at scale, within a global organization that values diversity, inclusion, and long-term client partnerships. Our Corporate Functions teams support the business across finance, risk, human resources, marketing, and other essential areas, and this is a full-time role.

last updated 36 week of 2026