Senior AI Platform Engineer – LLM Inference Backend

JPMorgan Chase & Co.; Greater London; On-site; GBP 110,000 - 150,000; Full time

Job summary

JPMorgan Chase & Co. is hiring a Software Engineer III in London to build backend services for LLM inference and scalable production systems.

You will work on routing, batching, scheduling, streaming responses, and quota management while improving APIs, observability, and reliability across the platform. You will explore model architectures, tokenization costs, and GPU utilization, collaborating with product teams and using enterprise AI tooling to boost performance and security of critical

Qualifications

• Formal training or certification on software engineering concepts and applied experience.
• Bachelor's Degree in Computer Science or equivalent.
• Solid programming fundamentals: data structures, concurrency basics, debugging, testing.
• Comfort with Go, Python, or TypeScript and ability to ramp up on others.
• Curiosity about LLMs and AI model architecture with willingness to learn quickly.
• Hands-on experience with enterprise-authorized AI-assisted software development tools with ability to evaluate outputs.

Responsibilities

• Build core backend services for LLM inference, including request routing, batching, scheduling, streaming responses, and quota/limits.
• Implement and maintain APIs and SDKs used by product and application teams across the firm.
• Profile and optimize performance end-to-end across CPU, memory, network, serialization, concurrency, GPU utilization, and caching.
• Improve reliability and operability through health checks, autoscaling, incident follow-ups, and runbooks.
• Contribute to system design by breaking down ambiguous problems and making tradeoffs.
• Add observability with metrics, tracing, logging, dashboards, and actionable alerts tied to SLOs.
• Support safe deployments through CI/CD improvements, canarying, feature flags, and backward compatibility.
• Learn LLM serving fundamentals — tokenization costs, KV cache, quantization, context length tradeoffs.
• Leverage enterprise AI tools and contribute reusable patterns to improve team effectiveness.

Skills

Go

Python

TypeScript

Distributed systems

Profiling/benchmarking

Unit testing

APIs (gRPC/HTTP)

Security practices

Debugging

Education

Bachelor's Degree in Computer Science or equivalent

Tools

Docker

Kubernetes

Git

CI/CD pipelines

Job description

JPMorgan Chase & Co. is hiring a Software Engineer III in London to build backend services for LLM inference and scalable production systems.

You will work on routing, batching, scheduling, streaming responses, and quota management while improving APIs, observability, and reliability across the platform. You will explore model architectures, tokenization costs, and GPU utilization, collaborating with product teams and using enterprise AI tooling to boost performance and security of critical