Lead Data Engineer - Lakehouse, Spark & AWS Expert

JPMorganChase; Jersey City (NJ); On-site; USD 140,000 - 210,000; Full time

Benefits offered by this job

Health coverage

Wellness centers onsite

Retirement savings plan

Tuition reimbursement

Mental health support

Job summary

JPMorgan Chase in New Jersey seeks a Lead Data Engineer within Corporate Technology to advance data collection, storage, and analytics with secure, scalable architectures. You will lead multi-team delivery, own critical pipelines, and shape data platforms across the firm.

Expertise in Databricks Lakehouse, Delta Lake, and AWS is required, plus leadership to drive design reviews, code quality, and cross-functional collaboration.

Qualifications

• 7+ years in data engineering with leadership experience on multi-team data platforms.
• Hands-on experience with Databricks Lakehouse on AWS.
• Deep Delta Lake expertise: ACID tables, partitioning, schema evolution.
• Proven Spark on Databricks: performance tuning, cluster sizing, skew mitigation.
• Experience with streaming and batch pipelines (Structured Streaming, backfills, late-arriving data).
• Strong AWS fundamentals for data platforms (S3, IAM, KMS, VPC, logging).
• Experience implementing data governance/security (Unity Catalog, permissions, credential passthrough).
• Ownership of reliability: monitoring/alerting, incident response, RCA, SLO/SLA.

Responsibilities

• Architect lake house designs (bronze/silver/gold or equivalent).
• Deliver scalable ingestion from AWS sources into Databricks (batch + streaming).
• Build maintainable pipelines with Delta Live Tables or standard Jobs; ensure testing and docs.
• Operational excellence: productionize workloads with Databricks Workflows/Jobs; implement retries and checkpointing.
• Governance by design: enforce least privilege, data classification, auditing, lineage.
• Performance & cost management: tune Spark/Delta workloads, right-size clusters, optimize storage.
• Lead and mentor: set engineering standards, drive code quality, upskill engineers in Spark/Databricks.
• Cross-functional delivery: translate stakeholder needs; align with security/platform teams.
• CI/CD and IaC: Terraform for Databricks + AWS; implement tests, replay/backfills.

Skills

Databricks Lakehouse

Spark on Databricks

Streaming & batch pipelines

AWS data platforms

Data governance & security

Reliability & SRE practices

CI/CD & IaC

Leadership

Cross-functional delivery

Tools

Databricks Workflows/Jobs

Delta Live Tables

Terraform

Job description

JPMorgan Chase in New Jersey seeks a Lead Data Engineer within Corporate Technology to advance data collection, storage, and analytics with secure, scalable architectures. You will lead multi-team delivery, own critical pipelines, and shape data platforms across the firm.

Expertise in Databricks Lakehouse, Delta Lake, and AWS is required, plus leadership to drive design reviews, code quality, and cross-functional collaboration.