Senior AI/ML Platform Engineer – Scale & Inference

JPMorganChase; Greater London; On-site; GBP 90,000 - 115,000; Full time

Job summary

JPMorganChase in London is seeking a Software Engineer III to design, build, and operate scalable LLM-serving services as part of the Firmwide platform. You will work in an agile team focused on performance, reliability, and secure production-grade code across a modern SDLC.

The role emphasizes learning model architectures, optimizing latency and throughput, and collaborating across product teams to deliver robust AI-powered capabilities at scale.

Qualifications

• Bachelor's Degree in Computer Science or equivalent.
• Solid programming fundamentals: data structures, concurrency basics, debugging, testing.
• Experience in Go, Python, or TypeScript with ability to ramp up others.
• Interest in distributed systems and system design.
• Curiosity about LLMs and AI model architecture.
• Measurement-driven mindset: profiling, benchmarking, data-driven improvements.
• Hands‑on experience with enterprise AI-assisted development tools.
• Understanding of responsible AI use and secure handling of data.

Responsibilities

• Build core backend services for LLM inference, including routing, batching, scheduling, streaming, and quotas.
• Implement and maintain APIs and SDKs used across the firm.
• Profile and optimize performance across CPU, memory, network, GPU, and caching.
• Improve reliability with health checks, autoscaling, incident follow-ups, and runbooks.
• Contribute to system design by breaking down problems and making tradeoffs.
• Add observability with metrics, tracing, logging, dashboards, and alerts tied to SLOs.
• Support safe deployments through CI/CD improvements, canaries, feature flags, and rollback plans.
• Learn LLM serving fundamentals like tokenization costs and KV cache.
• Leverage AI-assisted development tools and ensure outputs are correct, secure, and well-reviewed.

Education

Bachelor's Degree in Computer Science or equivalent

Tools

Docker

Kubernetes

gRPC

Redis

Job description

JPMorganChase in London is seeking a Software Engineer III to design, build, and operate scalable LLM-serving services as part of the Firmwide platform. You will work in an agile team focused on performance, reliability, and secure production-grade code across a modern SDLC.

The role emphasizes learning model architectures, optimizing latency and throughput, and collaborating across product teams to deliver robust AI-powered capabilities at scale.