Senior Databricks Lead & Data Platform Engineer

The Citco Group Limited; Dublin; On-site; EUR 120,000 - 160,000; Full time

Benefits offered by this job

Flexible working arrangements

Training and education support

Job summary

Citco is seeking a Team Lead - Data Engineer to direct a team of data engineers on the Centralized Data Platform built on Databricks, delivering scalable, secure, and high‑performing data solutions across Citco's global operations.

You will mentor engineers, shape best practices, and drive Databricks‑based Lakehouse architectures, real‑time pipelines, and Delta Live Tables, while collaborating across locations. A strong background in Python and Spark, plus cloud experience, is required.

Qualifications

• 8+ years of experience in data engineering.
• 3+ years hands-on Databricks experience.
• Proven leadership experience.
• Strong Python and Spark skills.
• Experience with AWS or other cloud services.
• Deep data modeling and SQL knowledge.
• Experience with Delta Lake and Lakehouse architecture.
• ETL/ELT principles and patterns.
• Version control with Git.
• Ability to optimize data pipelines.
• Excellent communication and collaboration.

Responsibilities

• Lead and mentor a team of data engineers, conduct reviews and knowledge-sharing sessions across locations.
• Drive the Agile/Scrum SDLC process and collaborate with team members.
• Design and develop Databricks solutions leveraging Lakehouse architecture for enterprise data processing and analytics.
• Develop and optimize ETL/ELT pipelines.
• Create and manage structured streaming pipelines for real-time data processing.
• Configure and optimize Databricks clusters and Spark jobs for performance.
• Utilize Delta Live Tables for data ingestion and transformations.
• Apply Unity Catalog features and IAM best practices for security governance.
• Support infrastructure and resource management using Terraform.
• Implement monitoring solutions for pipeline performance and data quality.
• Contribute to code reviews and knowledge-sharing sessions.

Skills

Data engineering

Databricks

Team leadership

Python

Spark

AI in development

AWS

Data modeling

SQL

Delta Lake

ETL/ELT design

Git

Data pipelines

Problem solving

Communication

Tools

Delta Lake

Unity Catalog

Git

Databricks

Job description

Citco is seeking a Team Lead - Data Engineer to direct a team of data engineers on the Centralized Data Platform built on Databricks, delivering scalable, secure, and high‑performing data solutions across Citco's global operations.

You will mentor engineers, shape best practices, and drive Databricks‑based Lakehouse architectures, real‑time pipelines, and Delta Live Tables, while collaborating across locations. A strong background in Python and Spark, plus cloud experience, is required.