Lead Data Engineer: Spark, Data Lakes & Risk Analytics

Testing Site - JPMC; Jersey City (NJ); On-site; USD 140,000 - 210,000; Full time

Job summary

JPMorgan Chase seeks a Lead Software Engineer to join the Market Risk MXL DataLake Team. You will design and implement high-volume data pipelines and historical data stores, partnering with architects, risk technologists, and product owners to push the boundaries of data platforms.

You will lead production-grade data engineering efforts, optimize Spark workloads, and advance platform standards while ensuring compliance with regulatory requirements.

Qualifications

• Degree-level education in Computer Science, Software Engineering, or related discipline.
• Strong software engineering fundamentals including data structures and system design.
• Proven experience building large-scale data engineering solutions on big-data platforms.
• Hands-on PySpark / Spark pipelines in production environments.
• Solid understanding of data modelling for analytical and historical data use cases.
• Experience working with large volumes of structured data over long time horizons.
• Familiarity with distributed systems concepts such as fault tolerance, parallelism, and idempotent processing.

Responsibilities

• Design, build, and maintain large-scale historical data stores on modern big-data platforms.
• Develop robust, scalable data pipelines using PySpark / Spark for batch and incremental processing.
• Apply strong data-modelling principles (e.g. dimensional, Data Vault-style, or similar) to support long-term historical analysis and regulatory requirements.
• Engineer high-quality production-grade code with a focus on correctness, performance, testability, and maintainability.
• Optimize Spark workloads for performance and cost efficiency (partitioning, clustering, file layout, etc.).
• Collaborate with architects and senior engineers to evolve platform standards, patterns, and best practices.
• Contribute to code reviews, technical design discussions, and continuous improvement of engineering practices.

Skills

PySpark / Spark

Data engineering

Distributed systems concepts

Data modelling

Big data platforms

Education

Degree-level education in Computer Science / Software Engineering

Tools

Databricks

Delta Lake

Job description

JPMorgan Chase seeks a Lead Software Engineer to join the Market Risk MXL DataLake Team. You will design and implement high-volume data pipelines and historical data stores, partnering with architects, risk technologists, and product owners to push the boundaries of data platforms.

You will lead production-grade data engineering efforts, optimize Spark workloads, and advance platform standards while ensuring compliance with regulatory requirements.