Lead Data Engineer

慨正橡扯; Greater London; On-site; GBP 110,000 - 150,000; Full time

Job summary

J.P. Morgan is seeking a Lead Data Engineer within Personal Investing to design, build, and operate a cloud-native data platform powering analytics, regulatory reporting, and data-driven products for UK investors.

You will mentor engineers, set technical direction, and collaborate with product and analytics teams while applying modern data engineering patterns and strong software fundamentals. This role emphasizes ownership, observability, security, and scalable pipelines using Python, PySpark,

Qualifications

• Degree in Computer Science or a STEM field (or equivalent).
• Demonstrated experience delivering in an agile, fast-paced engineering environment.
• 8 years of hands-on experience coding as a data engineer.
• Strong software engineering fundamentals and testing strategies.
• Cloud-based data platforms using major cloud services (AWS, Google Cloud, or Azure).
• Experience with large-scale distributed data processing and performance tuning.
• Experience with data warehousing/lakehouse technologies and engines (e.g., Spark, Trino).
• Strong SQL skills and dbt for transformations.
• Experience designing and operating orchestration pipelines (Airflow).
• Experience building streaming pipelines (Kafka, Pub/Sub).

Responsibilities

• Design scalable, reusable data processing frameworks using Python, PySpark, dbt.
• Build and optimize batch and streaming pipelines with high performance and observability.
• Develop and operate orchestration with Airflow to schedule and monitor data movement.
• Model and transform data for analytics using SQL and dbt.
• Write production-grade Python/PySpark code with testing and OOD design.
• Implement infrastructure-as-code (Terraform) to provision cloud components.
• Containerize and deploy services with Docker and Kubernetes.
• Collaborate with analysts, data scientists, and application teams to translate requirements into designs.
• Own critical data systems to improve reliability, security, and operations.
• Mentor junior engineers and influence technical direction through standards.

Skills

Python

PySpark

Software engineering

Agile methodologies

SQL

Testing

OO design

Cloud data platforms

Education

Bachelor's degree in Computer Science or STEM-related field

Tools

Airflow

dbt

Terraform

Docker

Kubernetes

Spark

Flink

Trino

Iceberg

Hudi

Redshift

BigQuery

Snowflake

Job description

JOB DESCRIPTION

Shape how hundreds of thousands of UK investors use data to make confident, informed investment decisions. Join a team building modern, cloud-native data platforms that enable analytics, regulatory reporting, and data-driven products at scale. Youâll work with contemporary lakehouse and streaming patterns, strong engineering practices, and a culture that values ownership and continuous improvement. This role offers meaningful scope to influence platform standards and mentor others while growing your technical and leadership impact.

Job summary

As a Lead Data Engineer at JPMorgan Chase within Personal Investing, you will design, build, and operate a robust cloud-native data platform and pipelines that power analytics, regulatory reporting, and data-promoten applications. You will help us deliver reliable, scalable, observable, and secure data solutions by applying strong software engineering fundamentals and modern data engineering patterns. Youâll work closely with partners across product, analytics, and engineering to translate business needs into resilient technical designs. Youâll also contribute to engineering excellence through best practices, mentoring, and thoughtful technical direction.

Job responsibilities

• Design scalable, reusable data processing and data quality frameworks using Python, PySpark, and dbt
• Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability
• Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations
• Model and transform data for analytics using SQL and dbt to support business intelligence and reporting workloads
• Write production-grade Python/PySpark code with disciplined testing, performance tuning, and maintainable object-oriented design
• Implement infrastructure-as-code (e.g., Terraform) to provision and manage cloud-based data platform components
• Containerize and deploy services using Docker and Kubernetes (and related tooling such as Helm)
• Collaborate with analysts, data scientists, and application teams to turn requirements into technical designs and delivered solutions
• Own critical data systems by improving reliability, scalability, security, and operational excellence
• Mentor junior engineers and influence the teamâs technical direction through standards, reviews, and knowledge sharing

Required qualifications, capabilities, and skills

• Degree in Computer Science or a STEM-related field (or equivalent)
• Demonstrated experience delivering in an agile, fast-paced engineering environment
• 8 years of recent, hands-on professional experience actively coding as a data engineer
• Strong software engineering fundamentals (system design, data structures, object-oriented programming, testing strategies, and end-to-end development lifecycle)
• Strong Python programming skills, including unit and integration testing
• Hands-on experience building and operating cloud-based data platforms using major cloud services (e.g., AWS, Google Cloud, or Azure)
• Experience with large-scale distributed data processing and performance tuning
• Hands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar)
• Strong SQL skills and experience with SQL-based transformation tooling (e.g., dbt)
• Experience designing and operating orchestration pipelines using Airflow or similar tools
• Experience designing and building streaming pipelines using Kafka, Pub/Sub, or similar messaging systems

Preferred qualifications, capabilities, and skills

• Data modeling experience for analytics and reporting use cases
• Knowledge of security, risk, compliance, and governance considerations for data platforms
• Experience building continuous integration and continuous delivery automation for data and platform services
• Experience with container-based deployment environments (Docker, Kubernetes, etc.)
• Demonstrated ability to coach teammates on engineering practices and contribute to a collaborative, inclusive team culture

ABOUT US

J.P. Morgan is a global leader in financial services, providing strategic advice and products to the worldâs most prominent corporations, governments, wealthy individuals and institutional investors. Our first-class business in a first-class way approach to serving clients drives everything we do. We strive to build trusted, long-term partnerships to help our clients achieve their business objectives.

We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicantsâ and employeesâ religious practices and beliefs, as well as mental health or physical disability needs. Visit our

FAQs

for more information about requesting an accommodation.

ABOUT THE TEAM

Our professionals in our Corporate Functions cover a diverse range of areas from finance and risk to human resources and marketing. Our corporate teams are an essential part of our company, ensuring that weâre setting our businesses, clients, customers and employees up for success.