Senior Data Engineer: Real-Time, Scalable Data Platform

Atlantic Search; Dublin; On-site; EUR 70,000 - 110,000; Full time

Job summary

Atlantic Search is seeking a Senior Data Engineer to help build a modern data platform for a long-term, multi-year funded programme of work. You will own the development of dependable data solutions and ensure data is delivered accurately and efficiently across the organisation.

The role focuses on data ingestion and streaming, real-time and batch processing, and improving scalability. You will work independently, make sound technical decisions and contribute to architectural discussions with

Qualifications

• Several years of commercial data engineering experience as a contractor.
• Strong understanding of data architecture, transformation patterns and distributed processing.
• Ability to work independently and deliver reliable, well-documented data solutions.

Responsibilities

• Develop and enhance data ingestion and transformation processes.
• Build solutions for both scheduled and real-time data flows.
• Improve the reliability, speed and scalability of data workloads.
• Monitor live data processes and resolve technical issues.
• Apply appropriate testing, documentation and coding standards.
• Contribute to technical discussions and solution planning.
• Work closely with colleagues across engineering, analytics and the wider business.

Skills

PySpark

Python

SQL

Tools

Azure Data Factory

Kafka

Event Hubs

Job description

Atlantic Search is seeking a Senior Data Engineer to help build a modern data platform for a long-term, multi-year funded programme of work. You will own the development of dependable data solutions and ensure data is delivered accurately and efficiently across the organisation.

The role focuses on data ingestion and streaming, real-time and batch processing, and improving scalability. You will work independently, make sound technical decisions and contribute to architectural discussions with