Senior Data Engineer - Lakehouse Pipelines & AI

G MASS Consulting; Dublin; On-site; EUR 90,000 - 130,000; Full time

Job summary

G MASS Consulting is recruiting a Senior Data Engineer for a leading global Financial Services client in Ireland. This hands-on role sits at the heart of a major data platform build with influence over architecture, pipelines, and AI initiatives across a high-volume environment.

You will design scalable data solutions on Lakehouse, build ETL/ELT pipelines, optimise Spark, and apply governance, IAM, and security practices within a robust AWS stack.

Qualifications

• 6+ years of data engineering experience with hands-on Databricks
• Strong Python and Spark programming skills
• Solid AWS experience across S3, Glue, Lambda
• Deep understanding of data modelling, SQL, and ETL/ELT design patterns
• Experience with Delta Lake and Lakehouse architecture
• Experience using AI tools within development workflows
• Excellent communication across technical and non-technical teams

Responsibilities

• Design and develop scalable data solutions on Lakehouse architecture
• Build and maintain ETL/ELT pipelines and streaming workflows
• Tune clusters and Spark jobs for performance at scale
• Use Delta Live Tables and Unity Catalog for ingestion and governance
• Follow IAM best practices and security standards
• Support infrastructure via Terraform
• Implement monitoring for pipelines and data quality
• Contribute to code reviews and documentation
• Collaborate with data scientists, analysts and stakeholders
• Explore AI tooling to boost development productivity

Skills

Databricks

Python

Spark

AWS

Data Modelling

ETL/ELT design

SQL

Git

Tools

Delta Lake

Lakehouse

Terraform

Unity Catalog

Git

Job description

We are working with a leading global Financial Services business to hire a Senior Data Engineer into their growing data engineering practice. This is a hands-on technical role sitting at the centre of a major enterprise data platform build, with scope to influence architecture, drive pipeline development, and contribute to AI initiatives across a complex, high-volume financial data environment.

Responsibilities:

• Design and develop scalable data solutions on a Lakehouse architecture platform, supporting enterprise-wide data processing and analytics
• Build, optimise, and maintain ETL/ELT pipelines and structured streaming workflows for both batch and real-time data ingestion
• Configure and tune clusters and Spark jobs to deliver consistent performance at scale
• Utilise Delta Live Tables and Unity Catalog to manage data ingestion, transformation, and access governance
• Apply IAM best practices and maintain compliance with data security standards across the platform
• Support infrastructure provisioning and resource management using Terraform
• Implement monitoring frameworks covering pipeline performance, data quality, and operational health
• Contribute to code reviews, technical documentation, and team knowledge-sharing
• Work within an Agile delivery model, collaborating closely with data scientists, analysts, and business stakeholders
• Explore emerging technologies and AI tooling to enhance development productivity and platform capability
  
  

Requirements

• 6+ years in data engineering, with at least 2 years hands-on experience with Databricks
• Strong Python and Spark programming skills
• Solid AWS experience across core services including S3, Glue, and Lambda
• Deep understanding of data modelling, SQL, and ETL/ELT design patterns
• Experience with Delta Lake, Lakehouse architecture, and Git-based version control
• Demonstrable use of AI tools within a professional development workflow
• Strong communication skills and the ability to work effectively across technical and non-technical teams
  
  

Desirable:

• Financial services or fund administration background
• Exposure to AI/ML implementation patterns and real-time data processing frameworks
• Multi-cloud experience beyond AWS
• API development or data governance framework experience
• Experience mentoring junior engineers
  
  

Benefits

Salary: to be discussed, depending on experience

Length: Permanent contract