Lead Data Engineer - Spark, PySpark & Lakehouse

Aumni; Jersey City (NJ); On-site; USD 160,000 - 230,000; Full time

Job summary

JPMorganChase, a global financial leader, seeks a Lead Software Engineer to advance data engineering and Spark-based ETL/ELT efforts in a secure, scalable lakehouse environment.

You will contribute as a core technical member, delivering production Python/PySpark code, optimizing Spark SQL queries, and guiding AI-assisted development across teams. Jersey City-based, with a focus on reliability and performance.

Qualifications

• Formal training or certification on software engineering concepts and 5+ years applied experience.
• 5+ years of applied experience building production data engineering and/or software engineering solutions.
• Hands-on experience delivering system design, application development, testing, and operational stability for large-scale data pipelines.
• Advanced in Python with PySpark experience and Spark SQL fundamentals.
• Experience leading AI-assisted development tools and establishing validation standards.

Responsibilities

• Develop secure, high-quality production code in Python/PySpark and Spark SQL.
• Lead adoption of enterprise AI-assisted engineering practices to improve quality and speed.
• Improve data pipelines, lakehouse maintenance, and data quality across deployments.
• Collaborate with vendors and internal teams on architectural designs and data platform relevance.
• Drive communities of practice around Spark performance and Iceberg operations.

Skills

Python

PySpark

Spark SQL

SQL

AI-assisted development tools

CI/CD

AWS data services

Databricks

Iceberg

Java

Education

Formal software engineering training or certification

Tools

Spark

EMR

Databricks

S3

AWS Glue Data Catalog

Job description

JPMorganChase, a global financial leader, seeks a Lead Software Engineer to advance data engineering and Spark-based ETL/ELT efforts in a secure, scalable lakehouse environment.

You will contribute as a core technical member, delivering production Python/PySpark code, optimizing Spark SQL queries, and guiding AI-assisted development across teams. Jersey City-based, with a focus on reliability and performance.