Databricks Tech Lead: Spark Pipelines & Cloud Platform

G MASS Consulting; Dublin; On-site; EUR 110,000 - 150,000; Full time

Job summary

G MASS Consulting seeks an experienced Databricks Technical Lead to own strategic data platforms within a financial services environment. You will define technical direction across Databricks and AWS, ensuring scalable and secure development practices throughout the full SDLC while collaborating with cross-functional teams across banking systems.

The role emphasizes architecture, data governance with Unity Catalog, and IaC with Terraform, plus CI/CD via Jenkins, mentoring engineers to deliver

Qualifications

• Enterprise SDLC experience required.
• Financial services exposure preferred.
• Proficient in Databricks (Spark/PySpark), Delta Lake, Unity Catalog.
• AWS experience (EC2, S3, IAM, etc.).
• Terraform / IaC and Jenkins CI/CD experience.

Responsibilities

• Lead development of banking data platforms on Databricks and AWS.
• Build and optimise large-scale data pipelines using Spark/PySpark, Delta Lake, and Databricks Workflows.
• Own data governance and Unity Catalog implementations.
• Manage cloud deployments with Terraform and IaC best practices.
• Maintain technical documentation and knowledge resources.
• Provide technical leadership through mentoring and code reviews.
• Drive CI/CD improvements across cloud/on‑prem environments.
• Promote clean coding, automated testing, and AI-assisted development.

Skills

Databricks

Spark

PySpark

Delta Lake

Unity Catalog

AWS

Terraform

Jenkins

Python

Scala

CI/CD

IBM MQ

Education

Degree in Computer Science or related discipline

Tools

Terraform

Job description

G MASS Consulting seeks an experienced Databricks Technical Lead to own strategic data platforms within a financial services environment. You will define technical direction across Databricks and AWS, ensuring scalable and secure development practices throughout the full SDLC while collaborating with cross-functional teams across banking systems.

The role emphasizes architecture, data governance with Unity Catalog, and IaC with Terraform, plus CI/CD via Jenkins, mentoring engineers to deliver