Lead Data Engineer - Databricks on AWS

Testing Site - JPMC; Wilmington (DE); On-site; USD 140,000 - 190,000; Full time

Job summary

JPMorganChase is seeking a Lead Data Engineer with deep Databricks and AWS experience to architect large-scale data pipelines and analytics platforms within the Corporate Technology Finance team.

You will lead a team of data engineers, collaborate with data scientists and business stakeholders, and ensure best practices in data engineering, security, and cloud architecture while driving performance, reliability, and cost efficiency.

Qualifications

• Formal training or certification on software engineering concepts and 5+ years applied experience.

Responsibilities

• Architect, develop, and optimize large-scale data pipelines and analytics platforms, leveraging Databricks (Spark, Delta Lake) and AWS cloud services.
• Lead a team of data engineers, collaborate with data scientists and business stakeholders, and ensure best practices in data engineering, security, and cloud architecture.
• Design, build, and maintain scalable ETL/ELT data pipelines using Databricks (Spark, Delta Lake) on AWS.
• Architect and implement data lake and data warehouse solutions leveraging AWS services (S3, Glue, Redshift, Lambda, EMR, etc.).
• Lead and mentor a team of data engineers, providing technical guidance and code reviews.
• Optimize data workflows for performance, reliability, and cost efficiency.
• Collaborate with data scientists, analysts, and business teams to deliver high-quality data products.
• Ensure data quality, security, and compliance with organizational and regulatory standards.
• Drive adoption of best practices in data modeling, version control, CI/CD, and infrastructure-as-code (e.g., Terraform, CloudFormation).
• Troubleshoot and resolve issues in production data pipelines and analytics platforms.

Skills

Databricks

Spark

Delta Lake

AWS

Python

Scala

SQL

CI/CD

DevOps

Terraform

CloudFormation

Leadership

Data modeling

Education

Software engineering certification

Tools

Terraform

CloudFormation

Job description

JPMorganChase is seeking a Lead Data Engineer with deep Databricks and AWS experience to architect large-scale data pipelines and analytics platforms within the Corporate Technology Finance team.

You will lead a team of data engineers, collaborate with data scientists and business stakeholders, and ensure best practices in data engineering, security, and cloud architecture while driving performance, reliability, and cost efficiency.