Software Engineer III, Databricks Data Pipelines

JPMorgan Chase & Co.; Kentucky; On-site; USD 110,000 - 170,000; Full time

Job summary

JPMorgan Chase & Co. in Kentucky is seeking a Software Engineer III within Employee and Experience Technology to design and deliver scalable data products, focusing on modernizing with a Databricks-on-AWS lakehouse.

You will build data pipelines with PySpark on Databricks, optimize performance, implement data quality with Delta Lake, and collaborate with BI partners using Sigma, Tableau, and Alteryx. Join a team that emphasizes secure, resilient systems and continuous improvement through

Qualifications

• 3+ years of data engineering experience.
• Advanced PySpark on Databricks for production pipelines.
• Strong AWS data ecosystem experience (S3, EMR, Glue, Lambda, Athena).
• Python programming for data processing.
• CI/CD proficiency with Git, Bitbucket, Jenkins or Spinnaker.
• Knowledge of agile methodologies, security, and operations.
• SQL and data modeling; Oracle is a plus.
• Experience with Airflow or Autosys for scheduling.

Responsibilities

• Design, build, and maintain data pipelines on Databricks using PySpark.
• Tune PySpark jobs and Databricks clusters for performance and cost.
• Implement scalable data frameworks with bronze/silver/gold lakehouse patterns.
• Apply Delta Lake and Delta Live Tables for data quality.
• Monitor ingestion issues with Databricks and AWS CloudWatch.
• Automate remediation using Databricks Workflows and AWS automation.
• Leverage AI tools to accelerate development.
• Deliver curated datasets to BI tools (Sigma, Tableau, Alteryx).
• Collaborate with stakeholders to design architecture artifacts.
• Contribute to engineering communities of practice.

Skills

Databricks

PySpark

AWS data ecosystem

Python

Data engineering

CI/CD

SQL/Data modeling

Airflow/Autosys

Security/operational stability

Scala/Java (plus)

Tools

Git/Bitbucket

Jenkins

Spinnaker

Job description

JPMorgan Chase & Co. in Kentucky is seeking a Software Engineer III within Employee and Experience Technology to design and deliver scalable data products, focusing on modernizing with a Databricks-on-AWS lakehouse.

You will build data pipelines with PySpark on Databricks, optimize performance, implement data quality with Delta Lake, and collaborate with BI partners using Sigma, Tableau, and Alteryx. Join a team that emphasizes secure, resilient systems and continuous improvement through