Lead Data Engineer: Scalable Pipelines & AI Solutions

JPMorgan Chase & Co.; Columbus (OH); On-site; USD 130,000 - 180,000; Full time

Job summary

JPMorgan Chase & Co. is seeking a Lead Software Engineer – Data Engineering within the Consumer & Community Banking/Data Products team.

You will serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure, scalable way. You will design, develop, and optimize large-scale ETL pipelines, build Python applications, and develop distributed data processing with PySpark while championing best practices, testing, and CI/CD across the product

Qualifications

• 5+ years of data engineering experience in a large corporate environment.
• 3+ years in hands-on Python (primary) / Java (secondary) development.
• Experience with PySpark and distributed data processing.
• Strong DBT and modern ETL development practices.
• Experience with workflow orchestration platforms (Control-M / Airflow).
• Expertise with AWS data services (Glue, Athena, Lambda, CloudWatch).
• Knowledge of Iceberg and/or Delta Lake.
• Snowflake administration and development experience.
• Strong SQL skills and modern database technologies.
• Experience with AI-assisted engineering practices and secure coding.

Responsibilities

• Design, develop, and optimize large-scale ETL data pipelines.
• Build high-quality Python applications with modular code, logging, and automated testing.
• Develop and maintain distributed data processing using PySpark.
• End-to-end testing design and validation at scale.
• Implement and support workflow orchestration using Control-M or Apache Airflow (MWAA).
• Develop cloud-native solutions leveraging AWS services (Glue, Athena, Lambda, CloudWatch).
• Design and manage modern data lake architectures using Iceberg/Delta Lake.
• Administer and optimize Snowflake environments (streams, tasks, roles, warehouses).
• Participate in code reviews and promote engineering best practices, testing standards, CI/CD.
• Leverage AI-assisted development tools while ensuring secure, compliant delivery.

Skills

Data engineering

Python

PySpark

SQL

Java

DBT

Airflow

Snowflake

AWS

Iceberg/Delta Lake

AI-assisted development

Education

Bachelor's degree in Computer Science or related field

Tools

Control-M

Apache Airflow

AWS Glue

AWS Athena

AWS Lambda

AWS CloudWatch

Snowflake

GitHub Copilot

Job description

JPMorgan Chase & Co. is seeking a Lead Software Engineer – Data Engineering within the Consumer & Community Banking/Data Products team.

You will serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure, scalable way. You will design, develop, and optimize large-scale ETL pipelines, build Python applications, and develop distributed data processing with PySpark while championing best practices, testing, and CI/CD across the product