Databricks Data Engineer III - Pipelines & AI Enablement

JPMorganChase; Dublin; On-site; EUR 70,000 - 110,000; Full time

Job summary

JPMorganChase in Ireland seeks a data engineer to design and operate scalable data pipelines for enterprise analytics. You will work with Databricks, Spark/PySpark, Python, and SQL to ingest, transform, and validate data assets across the data lifecycle.

You will collaborate with stakeholders in agile teams, apply CI/CD and testing, and help advance data governance, security, and AI-assisted development practices within the enterprise data platform.

Qualifications

• Hands-on experience with Databricks, Spark/PySpark, Python, and SQL.
• Experience developing and maintaining data pipelines and data processing systems.
• Understanding of the data lifecycle, including ingestion, transformation, storage, and consumption.
• Knowledge of cloud platforms (AWS) and distributed data processing.
• Experience with SDLC practices including CI/CD, testing, and deployment.
• Strong problem-solving skills and ability to troubleshoot data and pipeline issues.
• Ability to collaborate effectively within agile teams and across stakeholders.
• Hands-on experience using enterprise-authorized AI-assisted software development tools with validation.

Responsibilities

• Develop workflows and ELT data pipelines using Python, Spark/PySpark, and Databricks.
• Onboard enterprise datasets into Palmos, including ingestion, transformation, and validation.
• Build, test, and maintain scalable data pipelines and architectures for analytics.
• Apply best practices for performance, reliability, and maintainability.
• Support data security, governance, and entitlements frameworks.
• Use SQL extensively with relational and NoSQL data stores.
• Partner with stakeholders to translate data requirements into production solutions.
• Apply SDLC practices including CI/CD, testing, and monitoring.
• Contribute to reusable frameworks and standards.
• Identify data issues and optimization opportunities to improve quality and performance.
• Leverage enterprise-authorized AI coding assist tools while validating outputs via peer review & testing.

Skills

Databricks

Spark/PySpark

Python

SQL

CI/CD

Agile collaboration

Troubleshooting

AI-assisted development tools

Data governance & security

AWS

Palmos

Delta Lake

Tools

AWS

Palmos

Delta Lake

AI coding assist tools

Job description

JPMorganChase in Ireland seeks a data engineer to design and operate scalable data pipelines for enterprise analytics. You will work with Databricks, Spark/PySpark, Python, and SQL to ingest, transform, and validate data assets across the data lifecycle.

You will collaborate with stakeholders in agile teams, apply CI/CD and testing, and help advance data governance, security, and AI-assisted development practices within the enterprise data platform.