Senior Data Engineer - Databricks & Azure Lakehouse

Acuity; Cork; On-site; EUR 90,000 - 150,000; Full time

Job summary

Acuity Inc. in Cork, Ireland, is seeking a Senior Data Engineer to join the Digital Center of Excellence (DCoE).

You will design, build, and support trusted data pipelines and lakehouse data products using Databricks and Azure Data Lake to enable reporting, analytics, data science, AI, and downstream applications. The role demands hands-on execution, strong collaboration with architects, analysts, and business stakeholders, and a focus on production-quality ETL/ELT solutions, data quality, and

Qualifications

• Bachelor’s degree in computer science, information systems, engineering, or a related field, or equivalent practical experience.
• 5-6+ years of hands-on data engineering experience building production data pipelines and data products.
• Strong experience with Databricks, Azure Data Lake, Delta Lake, Python/PySpark, and SQL.
• Deep understanding of ETL/ELT, data integration, lakehouse architecture, and bronze/silver/gold design patterns.
• Experience with data quality, reconciliation, schema management, orchestration, performance tuning, and production support.
• Good understanding of reporting, analytics, and how curated data is consumed by Power BI or similar BI tools.

Responsibilities

• Design, build, and optimize scalable ETL/ELT pipelines using Databricks, Python/PySpark, SQL, Delta Lake, and Azure services.
• Ingest data from databases, APIs, files, SaaS applications, and event sources using appropriate batch or streaming patterns.
• Build bronze, silver, and gold datasets using reusable, metadata-driven, and configuration-driven engineering approaches.
• Implement incremental processing, schema evolution, error handling, restorability, reconciliation, and audit controls.
• Apply data-quality rules, testing, lineage, and monitoring to deliver trusted data products.
• Partner with architects, analysts, and business teams to define source‑to‑target mappings, business rules, and acceptance criteria.
• Curate performant datasets for Power BI, enterprise reporting, analytics, data science, AI, and downstream applications.
• Tune pipelines, SQL, Spark workloads, storage, and table design for reliability, performance, and cost.
• Use Git and Azure DevOps for backlog management, user stories, branching, pull requests, CI/CD, releases, and traceability.
• Create unit, integration, and reconciliation tests and participate in design and code reviews.
• Monitor production pipelines, resolve incidents, perform root‑cause analysis, and document operational procedures.
• Contribute reusable components, engineering standards, mentoring, and continuous improvement across the DCoE data team.

Skills

Databricks

Azure Data Lake

Delta Lake

Python/PySpark

SQL

ETL/ELT

lakehouse architecture

Power BI

Git

CI/CD

Agile delivery

Education

Bachelor’s degree in computer science or related field

Tools

Azure DevOps

Unity Catalog

Job description

Acuity Inc. in Cork, Ireland, is seeking a Senior Data Engineer to join the Digital Center of Excellence (DCoE).

You will design, build, and support trusted data pipelines and lakehouse data products using Databricks and Azure Data Lake to enable reporting, analytics, data science, AI, and downstream applications. The role demands hands-on execution, strong collaboration with architects, analysts, and business stakeholders, and a focus on production-quality ETL/ELT solutions, data quality, and