Databricks Tech Lead

Nicoll Curtin; Leinster; Hybrid; EUR 120,000 - 160,000; Full time

Job summary

Nicoll Curtin is seeking a Databricks Tech Lead in Dublin with a hands-on approach to data engineering and platform optimisation. You will lead technical design and development while remaining actively involved in coding, performance tuning and decision-making.

You will guide the team through scalable data pipelines, Spark workloads and Delta Lake implementations in a hybrid Dublin setting, collaborating with data engineers, architects and stakeholders to deliver robust solutions.

Qualifications

• Hands-on Databricks in production environments.
• Extensive data engineering experience with complex pipelines.
• Proficient in Python, PySpark and SQL.

Responsibilities

• Lead the design and development of scalable data engineering solutions with Databricks.
• Guide hands-on development across data pipelines and ETL/ELT workloads.
• Provide technical leadership, conduct code reviews and establish best practices.
• Optimize Spark workloads for performance, reliability and cost.
• Ensure data quality, governance, security and platform reliability.
• Collaborate with data engineers, architects and stakeholders to deliver robust solutions.
• Troubleshoot complex issues and drive continuous improvement.

Skills

Databricks

Python

PySpark

SQL

Apache Spark

Delta Lake

Lakehouse

Spark tuning

Team leadership

Tools

Azure

AWS

CI/CD

Job description

Databricks Tech Lead

Location: Dublin, Ireland (Hybrid)

We are looking for a hands‑on Databricks Tech Lead with strong engineering fundamentals and deep practical experience with Databricks. You will lead the technical design and development of data engineering solutions, while remaining actively involved in coding, platform optimisation and technical decision-making.

Key Responsibilities

• Lead the design, development and delivery of scalable data engineering solutions using Databricks.
• Drive hands‑on development across data pipelines, data processing and ETL/ELT workflows.
• Provide technical leadership, conduct code reviews and establish engineering best practices.
• Optimise data pipelines and Spark workloads for performance, reliability and cost.
• Ensure data quality, governance, security and platform reliability.
• Work closely with data engineers, architects and stakeholders to deliver robust solutions.
• Troubleshoot complex technical issues and support continuous improvement.

Key Requirements

• Strong hands‑on experience with Databricks in a production environment.
• Extensive data engineering experience, including building and maintaining complex data pipelines.
• Strong proficiency in Python, PySpark and SQL.
• Solid understanding of Apache Spark, distributed processing and performance optimisation.
• Experience with Delta Lake, lakehouse architecture and data pipeline orchestration.
• Experience with cloud data platforms, ideally Azure or AWS.
• Proven ability to lead engineers, make technical decisions and drive delivery.
• Strong problem‑solving skills and a focus on engineering quality.

Ideal Candidate

A technically strong data engineer who combines deep Databricks expertise with hands‑on development and the ability to lead a team. This is an engineering‑led role, so we need someone who can design, build, optimise and lead, rather than someone focused purely on architecture or people management.