Lead Data Engineer: AI & Big Data Pipelines

Mastercard; Dublin; On-site; EUR 70,000 - 90,000; Full time

Job summary

A global technology company is seeking a Lead Data Engineer to design and implement scalable data solutions for AI and ML across platforms. Responsibilities include ensuring data security and collaborating with various teams. Ideal candidates will have strong expertise in data engineering, security, and programming languages like Java and Python. A passionate individual will thrive in this role, contributing to a vision of an inclusive digital economy.

Qualifications

• Demonstrated experience in dealing with data engineering complexities.
• Proven expertise in leading real-time data pipelines.
• Hands-on experience with Hadoop, Apache Spark, Kafka, NiFi.

Responsibilities

• Design and implement scalable data engineering solutions.
• Ensure data security, integrity, and compliance.
• Collaborate with AI and product engineering teams.

Skills

Data engineering complexities of 5 Vs of Big data

Leading real-time data pipelines

Hadoop

Apache Spark

Kafka

NiFi

Data security

Cloud platforms (AWS)

Microservices architecture

Java

Python

Scala

Job description

A global technology company is seeking a Lead Data Engineer to design and implement scalable data solutions for AI and ML across platforms. Responsibilities include ensuring data security and collaborating with various teams. Ideal candidates will have strong expertise in data engineering, security, and programming languages like Java and Python. A passionate individual will thrive in this role, contributing to a vision of an inclusive digital economy.