Lead Data Engineer

henryschein; Dublin; On-site; EUR 120,000 - 180,000; Full time

Job summary

Henry Schein is seeking a highly technical Senior Data Engineer to design, build, and optimize scalable data pipelines and lakehouse platforms. You will lead a small offshore data engineering team and coordinate delivery of high‑quality data solutions.

The role requires hands‑on implementation, strong Python/Scala and SQL skills, and experience with Databricks/Spark, cloud platforms (Azure preferred), and data governance.

Qualifications

• 8-12 years of data engineering experience and hands-on coding.
• Proven track record coordinating offshore or distributed teams.
• Strong Python/Scala, SQL, and data pipeline optimization skills.
• Cloud data platforms experience, Azure preferred.

Responsibilities

• Design, develop, and optimize batch and real-time data pipelines using Databricks and Spark.
• Build scalable lakehouse architectures and high‑performance data models.
• Provide technical leadership and coordiante offshore delivery teams.
• Define tasks, review deliverables, and ensure timelines and standards.
• Lead ETL/ELT framework implementations and reusable patterns.
• Troubleshoot performance, data skew, partitioning, and large-scale processing.
• Implement data quality, lineage, observability, and governance controls.
• Collaborate with API, Analytics, and Platform teams to deliver production‑grade data assets.
• Promote best practices in coding, testing, and CI/CD for data pipelines.

Skills

Python

Scala

SQL

Data modeling

Performance tuning

Leadership

Tools

Databricks

Spark

Kafka/Confluent

Snowflake

Azure

Job description

Job Description

Role Summary

We are seeking a highly technical Senior Data Engineer to design, build, and optimize scalable data pipelines and platforms. This role is primarily hands‑on, with responsibility for leading and coordinating a small offshore data engineering team to deliver high-quality, performant data solutions.

Key Responsibilities

• Design, develop, and optimize complex batch and real‑time data pipelines using Databricks, Spark, and related technologies.
• Build and maintain scalable lakehouse architectures and high‑performance data models to support business driven outcomes
• Provide technical leadership and day‑to‑day coordination for a small offshore data engineering team.
• Define tasks, review deliverables, and ensure adherence to engineering standards and timelines.
• Lead the implementation of advanced ETL/ELT frameworks and reusable data engineering patterns.
• Solve complex data challenges including performance tuning, data skew, partitioning, and large‑scale processing.
• Implement robust data quality, validation, and observability frameworks.
• Collaborate closely with API, Analytics, and Platform teams to deliver reliable, production‑grade data assets.
• Drive best practices in coding standards, testing, and CI/CD for data pipelines.
• Troubleshoot and resolve production issues, ensuring high availability and reliability.
• Act as the primary liaison between onshore stakeholders and offshore delivery teams.

Required Qualifications

• 8-12 years of experience in data engineering.
• Proven experience leading or coordinating offshore or distributed engineering teams.
• Deep hands‑on expertise with Databricks, Spark, and distributed data processing.
• Advanced proficiency in Python and/or Scala, plus strong SQL skills.
• Strong experience with performance tuning and optimization of large‑scale data pipelines.
• Experience with cloud data platforms (Azure strongly preferred).
• Solid understanding of data modeling, streaming architectures, and data lifecycle management.
• Experience implementing data quality, lineage, and governance controls.

Preferred

• Experience with Kafka/Confluent and real‑time streaming pipelines.
• Exposure to Snowflake or similar cloud data warehouses.
• Experience supporting AI/ML pipelines or feature engineering workflows.
• Background in regulated industries (e.g., healthcare).

Henry Schein is committed to the principle of equal opportunities in employment in all spheres of its operation. Henry Schein UK Holdingsstrives to operate a policy of equal opportunity and not discriminate against any person gender, race, colour, nationality, ethnic or national origin, religion, sexual orientation, marital status, disability, age or any other characteristic protected by law.