Lead Data Engineer: Python & Databricks Pipelines

JPMorganChase; Glasgow; On-site; GBP 90,000 - 120,000; Full time

Job summary

JPMorganChase in the United Kingdom is seeking a Lead Data Engineer to design, build, and maintain scalable data pipelines and architectures within the Compute Infrastructure Platforms team. You will collaborate with agile teams to deliver secure, reliable data solutions that power the firm's analytics and decisions.

You will own end-to-end data lifecycle, from ingestion to access, implement backups and archiving, and champion data quality and security.

Qualifications

• Formal training or certification on data engineering concepts.
• Hands-on Python data pipeline development experience.
• Databricks proficiency for large-scale processing.
• Experience with relational and NoSQL databases.
• Experience with full data lifecycle: ingestion, transformation, storage, access.
• Experience implementing backup, recovery, archiving strategies.
• Strong data quality and RCA skills.

Responsibilities

• Design and deliver scalable, secure, and reliable data pipelines using Python and Databricks to support enterprise-level business needs.
• Develop and maintain data models and architectures that enable high-quality analytics and reporting across multiple business functions.
• Drive root cause analysis and corrective action for data quality issues, ensuring consumers can trust the data they rely on.
• Implement and manage database backup, recovery, and archiving strategies to ensure data availability and resilience.
• Evaluate and report on access control processes to determine the effectiveness of data asset security with minimal supervision.
• Collaborate with cross-functional teams to define data requirements and deliver solutions aligned with business objectives.
• Identify opportunities to optimize data workflows and improve overall pipeline performance and efficiency.
• Contribute to a team culture of diversity, opportunity, inclusion, and respect.

Skills

Python

Databricks

Data pipelines

Relational databases

NoSQL databases

Data quality

Data governance

Security & access

Education

Data engineering certification

Tools

Databricks

Database backup

Access control

Job description

JPMorganChase in the United Kingdom is seeking a Lead Data Engineer to design, build, and maintain scalable data pipelines and architectures within the Compute Infrastructure Platforms team. You will collaborate with agile teams to deliver secure, reliable data solutions that power the firm's analytics and decisions.

You will own end-to-end data lifecycle, from ingestion to access, implement backups and archiving, and champion data quality and security.