Lead Software Engineer - Production Support & SRE

Security Capital Research

Greater London

On-site

GBP 110,000 - 150,000

Full time

47 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

JP Morgan Chase seeks a Lead Software Engineer for Rates Production Management to own end-to-end production of front-office services, build automation, and AI-driven tooling. You will diagnose issues, drive remediation, and lead P1/P2 incident responses while upholding SRE practices across multi-region deployments.

You will design production-grade software in Python, contribute to critical systems, and collaborate with DevOps, business users, and infrastructure teams to improve resilience and

Qualifications

  • 7+ years' experience in front-office production support or production engineering.
  • Strong Python skills at production-code level - OO design, testing, and debugging distributed systems.
  • Hands-on AI/ML tooling experience for operational problems with responsible AI considerations.
  • Extensive monitoring/observability tooling and alerting framework knowledge.
  • Proven incident management capabilities with RCAs and post-mortem leadership.

Responsibilities

  • Own production for Rates front-office services including batch/EOD processing and real-time pricing.
  • Lead incident response and restoration across groups to accelerate recovery.
  • Perform post-incident RCAs and drive remediation to reduce recurrence and debt.
  • Maintain production governance, runbooks, and audit readiness.
  • Develop automation and self-healing tooling to reduce toil and MTTR.

Skills

Python production code
SRE principles
Incident management
Observability tooling
AI/ML tooling
Front-office rates
Batch processing
Distributed systems
CI/CD
Leadership

Education

Software engineering basics certification

Tools

Geneos
Grafana
Splunk
Dynatrace
OpenTelemetry
Oracle
PostgreSQL
Linux
Kubernetes
Docker
OpenShift
Terraform
CloudFormation

Job description

The CIB Rates Production Management Technology team comprises technical leaders and self-motivated engineers who are passionate about ensuring the stability, resilience, and continuous improvement of mission-critical front-office trading applications. The team operates as an engineering-led Production Management function - one that writes code, builds platforms, and applies AI to detect, diagnose, and remediate issues before they impact the business.

As a Lead Software Engineer within Rates Production Management, you will combine end-to-end production ownership of the Rates application estate with hands-on software engineering. You will design and build automation, observability, and AI-driven tooling that delivers early detection of batch, risk, and market data issues, accelerates root cause analysis, and enable safe, automated remediation. You will serve as a trusted technical authority during major incidents, production readiness reviews, and resiliency exercises, while championing Site Reliability Engineering (SRE) principles and the responsible adoption of AI across the team.

Key Responsibilities
  • Serve as the technical production owner for critical Rates front-office services - batch/EOD processing, real-time pricing, risk computations, trade management, and market data pipelines
  • Lead technical response and service restoration for P1/P2 incidents, coordinating across Application Development, Infrastructure, and business stakeholders to accelerate recovery
  • Lead post-incident reviews (RCAs/post-mortems), identify recurring failure patterns, and partner with development teams on root cause remediation to eliminate recurrence and reduce technical debt
  • Maintain strong production governance — access controls, operational procedures, runbooks, documentation, audit readiness, MEPC execution, and operational review processes
  • Design, build, and maintain production-grade software (primarily Python) for operational tooling, self-healing workflows, and automated runbooks that reduce manual toil and MTTR
  • Contribute code changes directly to Rates applications to fix production defects, improve performance, and harden resiliency, following the full SDLC - design, automated testing, peer review, and controlled release
  • Build reusable engineering components, APIs, and integrations (job schedulers, ticketing, messaging, data stores) that scale improvements across the CIB application portfolio
  • Design and deliver AI/ML-based early-detection capabilities - anomaly detection on batch timings, job volumes, risk outputs, and market data, with predictive alerting on SLA breaches before business impact
  • Define and improve SLIs, SLOs, error budgets, and reliability metrics for the Rates application suite, reporting production health to senior technology and business leadership
  • Design and deploy monitoring, telemetry, dashboards, and alerting (Geneos, OpenTelemetry, Grafana, or equivalent)
  • Drive capacity planning, performance profiling, resiliency testing, and BCP/failover readiness across multi-region deployments
  • Represent Production Management in architecture and production readiness reviews for new features, platform changes, and cloud migration
  • Build and maintain strong relationships with Application Development, DevOps, Front Office business users, and GTI/Infrastructure teams
  • Communicate effectively with all levels of management - from technical deep-dives to executive-level summaries of production health and automation impact

Technical Expertise (Required)
  • Experience: 7+ years' experience in front-office production support, SRE, or production engineering within Investment Banking, preferably supporting Fixed Income Rates trading applications (pricing, risk, trade management, EOD/batch processing)
  • Software Engineering: Formal training or certification in software engineering concepts; strong Python skills at production-code level - object-oriented design, testing, debugging complex distributed systems, and building automation tooling within a full SDLC with code review and CI/CD
  • AI & Automation: Hands-on experience applying AI/ML or LLM-based tooling to operational problems (anomaly detection, triage, RCA, automated remediation, reporting) and leading effective use of AI-assisted development tools, with a strong understanding of responsible AI - data sensitivity, guardrails, and validation of outputs
  • Monitoring & Observability: Extensive experience with Geneos and/or Grafana, Splunk, Dynatrace, OpenTelemetry, log aggregation, and alerting frameworks
  • Databases: Strong SQL (Oracle, Postgres) and query performance tuning in high-throughput environments
  • Platform Knowledge: Linux/UNIX (RHEL), shell scripting (bash/ksh), job scheduling systems, and batch orchestration
  • Incident Management: Proven experience leading complex P1 incidents, managing war rooms, problem management, and producing actionable RCAs that eliminate recurrence
Technical Expertise (Preferred)
  • Knowledge of Rates products (Swaps, Options, Inflation, Bonds/Futures, Repo/Securities Lending) and the front-to-back trade lifecycle
  • Experience building agentic AI workflows, prompt engineering, or integrating LLMs with internal tooling (e.g. MCP servers, knowledge graphs, retrieval-augmented generation)
  • Hands-on experience with public cloud (AWS), infrastructure-as-code (Terraform, CloudFormation), and containerisation (OpenShift, Kubernetes, Docker)
  • Experience with time-series/statistical analysis and data libraries for building data-driven detection and dashboards
  • Experience with capacity management, performance profiling, and memory analysis in Python/Java environments

General Competencies
  • Proven ability to lead under pressure - managing multiple critical incidents simultaneously while maintaining clear communication
  • Engineering mindset - automates repeatable work, measures outcomes, and treats operational problems as software problems
  • Strategic thinker with the ability to balance immediate production needs against long-term platform and automation improvements
  • Strong ownership mentality with a track record of driving initiatives from conception to measurable outcomes (e.g. reduced MTTR, fewer incidents, improved SLA adherence)
  • Excellent communication skills (verbal and written) - able to translate complex technical issues into business impact language
  • Curiosity and continuous learning, particularly in emerging AI technologies, applied with sound judgement on risk and control
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Software Engineer - Production Support & SRE
Lead Software Engineer - Production Support & SRE

Aumni • Greater London

On-site
GBP 120,000 - 150,000
Python Front Office Rates - Lead Software Engineer
Python Front Office Rates - Lead Software Engineer

JPMorganChase • Greater London

On-site
GBP 120,000 - 180,000
Front Office Production Engineer
Front Office Production Engineer

Morson Edge (Financial Services) • City Of London

On-site
GBP 70,000 - 90,000
Production Ai Engineer - Vice President
Production Ai Engineer - Vice President

11037 Citibank, N.A. United Kingdom • England

Hybrid
GBP 90,000 - 130,000
27 days annual leave
Discretional annual bonus
Private medical care & life insurance
+5
Site Reliability Engineer
Site Reliability Engineer

Autonomai Recruitment • Greater London

On-site
GBP 300,000 - 500,000
Vice President, Production Services Application Support
Vice President, Production Services Application Support

BNY • Greater London

On-site
GBP 250,000 - 350,000
Senior Lead Software Engineer - Python
Senior Lead Software Engineer - Python

JPMorgan Chase & Co. • Auchentibber

On-site
GBP 90,000 - 150,000
Senior Lead Software Engineer - Python
Senior Lead Software Engineer - Python

JP Morgan Chase • Glasgow

On-site
GBP 110,000 - 190,000
Senior Lead Software Engineer - Athena Rates Risk and PNL
Senior Lead Software Engineer - Athena Rates Risk and PNL

Testing Site - JPMC • Greater London

On-site
GBP 120,000 - 150,000
Lead Rates Production Engineer — AI, Automation & SRE
Lead Rates Production Engineer — AI, Automation & SRE

Aumni • Greater London

On-site
GBP 120,000 - 150,000