Data Quality Engineer: Databricks & Lakehouse Expert

fulcrumdigital; Dublin; On-site; EUR 80,000 - 120,000; Full time

Job summary

fulcrumdigital is seeking a highly skilled Data Quality Engineer with strong Data Engineering expertise to ensure the accuracy, reliability, and scalability of enterprise data platforms in cloud environments.

You will validate data across large-scale pipelines using Databricks, PySpark, Hadoop, Hive, and SQL, build quality frameworks, and collaborate with data engineers and analysts to ensure reliable data delivery in Lakehouse architectures.

Qualifications

• 3+ years of experience in data engineering or data testing.
• Hands-on experience with Databricks and PySpark.
• Strong experience with Hive, HDFS, Spark and related Apache technologies.
• Advanced SQL for large-scale data validation and analysis.
• Experience with Data Warehouses, Data Lakes and Lakehouse architectures.
• Understanding of Star Schema, Snowflake Schema, and dimensional modeling.
• Cloud platforms experience (Azure, AWS, or GCP).

Responsibilities

• End-to-end validation of data pipelines across ingestion, transformation and consumption layers.
• Source-to-target reconciliation and data quality checks.
• Identify, investigate and resolve data anomalies and inconsistencies.
• Define and implement data quality frameworks, metrics and controls.
• Develop and validate data pipelines using PySpark and Databricks.
• Support ETL/ELT workflows and ensure data integrity throughout the lifecycle.
• Collaborate with Data Engineers, Architects and Analysts.

Skills

Databricks

PySpark

SQL

Data Modeling

Cloud Platforms

Data Warehouses/Lakes/Lakehouse

Star/Snowflake Schema

Dimensional Modeling

Hadoop Ecosystem

Data Quality & Validation

Tools

Databricks

PySpark

Hadoop

Hive

HDFS

Spark

Delta Lake

Unity Catalog

Airflow

Kafka

Job description

fulcrumdigital is seeking a highly skilled Data Quality Engineer with strong Data Engineering expertise to ensure the accuracy, reliability, and scalability of enterprise data platforms in cloud environments.

You will validate data across large-scale pipelines using Databricks, PySpark, Hadoop, Hive, and SQL, build quality frameworks, and collaborate with data engineers and analysts to ensure reliable data delivery in Lakehouse architectures.