Lead Data Engineer

DataJobs; Chicago (IL); On-site; USD 133,000 - 175,000; Full time

Job summary

JPMorganChase in Chicago, IL is seeking a Lead Data Engineer to design, implement, and maintain data pipelines and secure data architectures for the Corporate Technology team. The role focuses on timeliness, data quality, and regulatory compliance, with end-to-end solution delivery and influence over technical strategy.

The candidate will work with Java, Python, SQL, Spark, Kafka and cloud services across GCP/AWS, ensuring robust data governance, observability, and performance at scale.

Qualifications

• Experience designing, building and optimizing production ETL/ELT pipelines (batch and streaming).
• Strong Java and Python skills with testing and code review practices.
• Proficient SQL with joins, aggregations, subqueries, and window functions.

Responsibilities

• Design and develop scalable, secure distributed architectures for data ingestion and processing using cloud-native tech.
• Design, implement, and maintain data pipelines that collect, process, and store large data volumes with timeliness, quality and completeness.
• Ensure compliance with data residency and privacy regulations and apply security best practices for data in transit and at rest.

Skills

Java

Python

SQL

Spark

Flink

Dataflow

Kafka

schema registry

data modelling

partitioning

clustering

Snowflake

Databricks

cloud storage

Tools

Delta

Iceberg

Avro

Parquet

Job description

JPMorganChase is hiring a Lead Data Engineer within its Corporate Technology team in Chicago, IL on an onsite basis. In this role, you will design, implement, and maintain data pipelines that support scalable data ingestion and processing, with a strong focus on timeliness, quality, completeness, and compliance.

You will build secure, production-grade data architectures and collaborate with technical teams and business stakeholders to shape approaches that meet current and future needs. The position also involves setting a technical target state for the product and driving progress against the strategy, while evaluating emerging technologies and delivering solutions through end-to-end design and development.

What you'll do

• Design and develop scalable, secure distributed architectures for data ingestion and processing, leveraging cloud-native technologies and services.
• Design, implement, and maintain data pipelines that collect, process, and store large volumes of data from multiple sources while ensuring timeliness, quality, and completeness.
• Ensure compliance with data residency and privacy regulations, and apply security best practices for data in transit and at rest in line with financial regulations and firm-wide policies.
• Engage technical teams and business stakeholders to discuss and propose technical approaches.
• Define the technical target state of the product and drive achievement of the overall strategy.
• Evaluate technology recommendations and provide feedback on new tools and approaches.
• Execute creative software solutions across design and development work.

What you bring

• Comfort with Java and Python, including sound testing and code review practices.
• SQL expertise, including joins, aggregations, subqueries, and window functions.
• Experience designing, building, and optimizing production ETL/ELT pipelines (batch and streaming) using frameworks such as Spark, Flink, Dataflow, or similar.
• Hands-on experience with Kafka, including topics, keys, partitions, consumer groups, and at-least-once semantics, plus basic knowledge of schema registry.
• Practical knowledge of data modelling, partitioning, and clustering.
• Hands-on experience with Snowflake, Databricks, or similar, along with cloud storage or HDFS.
• Production experience with at least one major cloud provider (GCP or AWS) using native data services, with awareness of FinOps and cost-effective design.
• Experience implementing data quality checks, backfills, and incorporating SLIs with observability and reporting across lakehouse platforms and table formats such as Delta, Iceberg, Avro, and Parquet, including time-travel.

Preferred qualifications

• Experience with Kafka, Flink, or other streaming technologies.
• Familiarity with AI/ML technologies, including LLMs, prompt engineering, vector search, and responsible AI, along with experience using AI-assisted software development tools such as GitHub Copilot or Claude.
• Financial services industry experience and understanding of large-scale enterprise data environments.
• Experience mentoring engineers and leading technical delivery initiatives.

Compensation and location

• Location: Chicago, IL (onsite)
• Salary: USD 133,000 - 175,000 per year

Technologies: Java, Python, SQL, Spark, Flink, Dataflow, Kafka, schema registry, Snowflake, Databricks, HDFS, GCP, AWS, Delta, Iceberg, Avro, Parquet