Senior Data Engineer: Scalable Pipelines & Real-Time Analytics

Prudential Annuities Distributors (PAD); Letterkenny; On-site; EUR 90,000 - 120,000; Full time

Job summary

PGIM Ireland seeks a Senior Data Engineer to design, build, test, and maintain data pipelines powering platform delivery. You will work across data engineers, analysts and architects to translate complex business problems into scalable solutions using Python, SQL and Spark, with Microsoft Fabric as a core platform.

You will apply modern engineering practices, write automated tests, and collaborate in an agile environment, delivering reliable data services while continually learning new

Qualifications

• Bachelor's degree in Computer Science or Engineering or related field.
• Experience with DevOps automation tools and SDLC knowledge.
• Proven ability to learn new skills and collaborate across teams.
• Strong problem solving and communication, with agile working experience.

Responsibilities

• Build and optimize data pipelines using Python, SQL and Spark on MS Fabric.
• Develop high-quality, well-documented code meeting Prudential standards.
• Analyze data, prepare sources for modeling and insights across platforms.
• Collaborate with analysts, scientists and architects to improve data processes.
• Write unit/integration tests and automate quality checks.
• Apply emerging technologies and coach others to embed learning.
• Work on complex problems requiring in-depth evaluation of factors.

Skills

Python

SQL

Scala

Pyspark/Apache Spark

Shell scripting

Microsoft Fabric

Airflow

Tableau/Power BI

Data pipelines

Education

Bachelor of Computer Science or Engineering

Tools

Microsoft Fabric

Airflow

AWS

SQL

NoSQL/Graph DBs

Job description

PGIM Ireland seeks a Senior Data Engineer to design, build, test, and maintain data pipelines powering platform delivery. You will work across data engineers, analysts and architects to translate complex business problems into scalable solutions using Python, SQL and Spark, with Microsoft Fabric as a core platform.

You will apply modern engineering practices, write automated tests, and collaborate in an agile environment, delivering reliable data services while continually learning new