Senior Data Engineer

Atlantic Search; Dublin; On-site; EUR 70,000 - 110,000; Full time

Job summary

Atlantic Search is seeking a Senior Data Engineer to help build a modern data platform for a long-term, multi-year funded programme of work. You will own the development of dependable data solutions and ensure data is delivered accurately and efficiently across the organisation.

The role focuses on data ingestion and streaming, real-time and batch processing, and improving scalability. You will work independently, make sound technical decisions and contribute to architectural discussions with

Qualifications

• Several years of commercial data engineering experience as a contractor.
• Strong understanding of data architecture, transformation patterns and distributed processing.
• Ability to work independently and deliver reliable, well-documented data solutions.

Responsibilities

• Develop and enhance data ingestion and transformation processes.
• Build solutions for both scheduled and real-time data flows.
• Improve the reliability, speed and scalability of data workloads.
• Monitor live data processes and resolve technical issues.
• Apply appropriate testing, documentation and coding standards.
• Contribute to technical discussions and solution planning.
• Work closely with colleagues across engineering, analytics and the wider business.

Skills

PySpark

Python

SQL

Tools

Azure Data Factory

Kafka

Event Hubs

Job description

A Senior Data Engineer is required to support the development of a modern data platform for a long-term, multi-year funded programme of work.

You will take ownership of building dependable data solutions, improving existing processes and helping ensure data is delivered accurately and efficiently across the organisation.

The Role

• Develop and enhance data ingestion and transformation processes
• Build solutions for both scheduled and real-time data flows
• Improve the reliability, speed and scalability of data workloads
• Monitor live data processes and resolve technical issues
• Apply appropriate testing, documentation and coding standards
• Contribute to technical discussions and solution planning
• Work closely with colleagues across engineering, analytics and the wider business

Core Skills

• PySpark
• Azure Data Factory or comparable orchestration technology
• Python
• SQL
• Kafka, Event Hubs or another streaming platform

Applicants should have several years of commercial data engineering experience as a contractor and a strong understanding of data architecture, transformation patterns and distributed processing.

The role will suit an engineer who can work independently, make sound technical decisions and deliver straightforward solutions that are reliable and easy to support.