Senior Data Engineer

GCS Recruitment; Ireland; On-site; EUR 90,000 - 120,000; Full time

Job summary

GCS Recruitment Specialists, on behalf of our client, seeks a Senior Data Engineer in Dublin. This hybrid, 12-month contract focuses on building and maintaining ETL pipelines into data warehouses and data lakes.

You will design scalable data solutions using Azure Databricks, Snowflake, Airflow, and SQL, ensuring data quality in a regulated environment and collaborating with analytics and product teams to enable reporting and ML use cases.

Qualifications

• Experience designing data solutions with architectural patterns and data modelling.
• Hands-on data processing jobs with PySpark/SQL and strong software engineering principles.
• Experience with Azure Databricks, Snowflake, and orchestrating pipelines with Airflow or similar.
• Proficiency in SQL, relational databases, and data governance in regulated environments.

Responsibilities

• Design and maintain ETL pipelines from multiple sources to data warehouses/data lakes.
• Develop data transformation routines, handle complex data structures, and ensure data quality.
• Contribute to frameworks for code deployment, automation, and monitoring of pipelines.
• Collaborate with analytics, product, and infrastructure teams to productionise pipelines.
• Implement data governance, masking, and de-identification per regulatory standards.

Skills

ETL design

PySpark/SQL

Azure Databricks

Snowflake

Airflow

CI/CD pipelines

Data governance

SQL optimization

Cloud data engineering

Communication

Tools

Azure Data Factory

Airflow

Apache Spark

Snowflake

Azure Databricks

SQL Server

Git

Job description

Senior Data Engineer

Dublin, Ireland | Hybrid | 12-month Contract

About the Company

Our client is a large, global technology-driven organisation operating in a highly regulated, data-rich industry, delivering services at scale to millions of customers worldwide.

About the Role

As a Senior Data Engineer, you will develop and maintain data pipelines that extract, transform, and load (ETL) data from a wide variety of sources into centralised data platforms such as a data warehouse or data lake. You will ensure the smooth, reliable flow of data from source to destination systems while upholding rigorous data quality and integrity standards, working within a highly regulated environment.

Primary Responsibilities

• Data integration: integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers. Handle ingestion, transformation, and consolidation to create a unified, reliable data foundation for analysis and reporting.
• Data transformation and processing: develop transformation routines to clean, normalise, and aggregate data. Handle complex data structures and missing or inconsistent data, preparing data for analysis, reporting, and machine learning use cases.
• Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines.
• Implement data governance and data de-identification frameworks in line with company and regulatory standards.
• Partner with data analytics and product leaders to design best practices and standards for developing and productionising analytic pipelines.
• Partner with infrastructure leaders on architecture approaches to advance the data and analytics platform, including exploring new cloud-based tools and techniques (Azure, Snowflake, and others).
• Monitoring and support: monitor data pipelines and systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling to ensure data integrity and system reliability.

Required Qualifications

• Extensive experience designing data solutions, with a solid understanding of architectural patterns, system design principles, data modelling, and large-scale solution development.
• Extensive hands-on experience developing data processing jobs (PySpark/SQL) demonstrating strong software engineering principles.
• Proven experience with Azure Databricks and Snowflake.
• Experience orchestrating data pipelines using tools such as Azure Data Factory, Airflow, or similar.
• Proven experience working with Spark.
• Advanced SQL skills and deep experience with relational databases (e.g., SQL Server), including optimisation, stored procedures, and performance tuning for high-volume applications.
• Experience with DevOps tooling, Git workflows, and building CI/CD pipelines.
• Experience applying data governance controls, data masking, and data de-identification rules within a highly regulated environment.
• Excellent communication and interpersonal skills, with the ability to convey technical concepts to both technical and non-technical stakeholders.

Preferred Qualifications

• Experience analysing and improving workflows: evaluating process maps and flowcharts, identifying inefficiencies, and recommending enhancements to procedures, technology, and tools.
• Familiarity with languages such as Java or Scala.
• A motivated self-starter who manages their own tasks and takes ownership.
• Experience working in agile/scrum teams.
• Strong Python skills for data manipulation, cleaning, transformation, and analysis of structured data.
• Familiarity with production-quality ML/AI model development and deployment.
• Data architecture experience.

GCS is acting as an Employment Business in relation to this vacancy.