Lead Data Engineer — Scale Data Pipelines & Platforms

SpotOn TA; Dublin; On-site; EUR 120,000 - 150,000; Full time

Benefits offered by this job

Office based Dublin

Career progression

Global team

Dynamic work environment

Job summary

SpotOn TA in Dublin, Ireland is seeking a Data Engineering Lead to collaborate with Data Scientists, Product Managers, and Engineering teams. This role focuses on leading data engineers, building scalable data pipelines, and ensuring reliable data platforms for analytics and ML use cases.

The position emphasizes cross-functional collaboration, agile practices, and driving data architecture improvements to support growth in a dynamic, international tech environment.

Qualifications

• Fluent English at high proficiency is required.
• Strong SQL and experience with relational and NoSQL databases.
• Hands-on experience with Python, Scala, or Java.
• Experience with Databricks and Apache Spark/PySpark.
• Solid understanding of distributed systems and data architecture.
• Experience with cloud-native data platforms (AWS or Azure).
• Minimum 8+ years in Data Engineering or Big Data environments.
• Bachelor's degree in CS/Software Engineering or related field.

Responsibilities

• Lead and mentor data engineers across projects and squads.
• Integrate data from multiple structured and unstructured sources into unified platforms.
• Ensure best practices in engineering design, coding standards, and system reliability.
• Support production-grade deployment of data engineering solutions.
• Promote agile delivery and collaborative engineering practices.
• Build and optimise large-scale data pipelines for analytics and ML use cases.
• Define and evolve data architecture to support scalability, reliability, and performance.

Skills

Fluency in English (C1/C2)

SQL skills

Python/Scala/Java

Distributed systems

Cloud platforms (AWS/Azure)

NoSQL databases

Data architecture

Education

Degree in Computer Science or Software Engineering

Tools

Databricks

Apache Spark/PySpark

Hadoop ecosystem (Hive/Impala/Oozie/Sqoop/NiFi)

Job description

SpotOn TA in Dublin, Ireland is seeking a Data Engineering Lead to collaborate with Data Scientists, Product Managers, and Engineering teams. This role focuses on leading data engineers, building scalable data pipelines, and ensuring reliable data platforms for analytics and ML use cases.

The position emphasizes cross-functional collaboration, agile practices, and driving data architecture improvements to support growth in a dynamic, international tech environment.