Senior Data Engineer — Databricks on AWS Lakehouse (PySpark)

DataJobs; Columbus (OH); On-site; USD 110,000 - 150,000; Full time

Benefits offered by this job

Competitive total rewards

On-site health and wellness centers

Retirement savings plan

Tuition reimbursement

Mental health support

Diversity and inclusion

Job summary

JPMorganChase is modernizing data platforms with a Databricks‑on‑AWS lakehouse. You will design and deliver production‑grade PySpark pipelines, optimize for performance and cost, and enable trusted analytics for business partners.

You’ll work on end‑to‑end Databricks pipelines, implement data quality with Delta Lake, and collaborate across teams to solve complex data problems. The role emphasizes inclusion, diverse teams, and growth through communities of practice.

Qualifications

• 3+ years of data engineering experience with software engineering concepts.
• Advanced PySpark experience for large‑scale processing on Databricks.
• Strong AWS data ecosystem knowledge (S3, EMR, Glue, Lambda, Athena).
• Solid Python skills; Java/Scala a plus.
• Experience with CI/CD and automation; familiar with Git/Bitbucket/Jenkins.

Responsibilities

• Design, build, and maintain new data pipelines on Databricks with PySpark.
• Optimize PySpark jobs and Databricks clusters for performance and cost.
• Create scalable data frameworks for end‑to‑end Databricks pipelines.
• Implement data quality checks using Delta Lake and Delta Live Tables.
• Build monitoring and alerting via Databricks and AWS CloudWatch.
• Automate remediation using Databricks Workflows and AWS automation.
• Apply AI tooling (Claude, GitHub Copilot) to accelerate development.
• Deliver curated datasets for BI partners using Sigma, Tableau, Alteryx.
• Collaborate with stakeholders to design architecture artifacts.
• Contribute to engineering communities of practice focusing on new tech.

Skills

PySpark/ Spark

Databricks

Python

SQL

Airflow

CI/CD

System design

Data modeling

Cloud fundamentals

Performance tuning

Tools

Databricks

Delta Lake

Delta Live Tables

AWS CloudWatch

Sigma

Tableau

Alteryx

Python

Java

Scala

Git

Bitbucket

Jenkins

Spinnaker

S3

EMR

Glue

Lambda

Athena

Parquet

Iceberg

SQL

Oracle

Airflow

Autosys

Job description

JPMorganChase is modernizing data platforms with a Databricks‑on‑AWS lakehouse. You will design and deliver production‑grade PySpark pipelines, optimize for performance and cost, and enable trusted analytics for business partners.

You’ll work on end‑to‑end Databricks pipelines, implement data quality with Delta Lake, and collaborate across teams to solve complex data problems. The role emphasizes inclusion, diverse teams, and growth through communities of practice.