Data Quality Engineer: Azure Data & Lakehouse Expert

PLATO; Canada; On-site; CAD 85,000 - 120,000; Full time

Job summary

PLATO seeks a detail-oriented Data Quality Engineer to ensure accuracy and integrity of data across cloud platforms. You will validate ingestion, transformation, and movement across Bronze to Gold layers, ensuring business rules and lineage are met.

The ideal candidate brings strong ETL/ELT testing experience, SQL proficiency, and hands-on work with Azure data technologies such as Databricks, ADLS, ADF, Delta Lake, Kafka, and DevOps. Collaborate with teams in a fast-paced data-driven environment.

Qualifications

• Experience testing data pipelines, ETL/ELT processes, data warehouses, data lakes, or lakehouse solutions.
• Strong knowledge of data validation techniques, data quality controls, and source-to-target mapping.
• Experience validating structured and semi-structured data.
• Advanced SQL skills for data analysis, reconciliation, and defect investigation.
• Working knowledge of Python and PySpark for data testing and validation activities.
• Familiarity with data profiling and data quality tools.
• Familiarity with Azure data technologies including Azure Databricks, ADLS, ADF, Delta Lake, Kafka or Event Hub, and Azure DevOps.

Responsibilities

• Validate data ingestion, transformation, and movement across Bronze, Silver, and Gold data layers.
• Test data completeness, accuracy, conformity, lineage, business rules, and exception handling.
• Perform ETL/ELT, data warehouse, data lake, and lakehouse testing.
• Validate source-to-target mappings, schema changes, and transformation logic.
• Verify batch, incremental, and CDC processing.
• Conduct large-volume data validation and reconciliation activities.
• Assess data quality, auditability, and end-to-end data lineage.
• Investigate and troubleshoot data defects using SQL and other analysis tools.
• Collaborate with engineering and business teams to ensure data solutions meet functional and quality requirements.
• Support test planning, defect tracking, and reporting activities using Azure DevOps or similar tools.

Tools

Azure Databricks

Azure Data Lake Storage

Azure Data Factory

Delta Lake

Kafka

Event Hub

Azure DevOps

Python

PySpark

SQL

Job description

PLATO seeks a detail-oriented Data Quality Engineer to ensure accuracy and integrity of data across cloud platforms. You will validate ingestion, transformation, and movement across Bronze to Gold layers, ensuring business rules and lineage are met.

The ideal candidate brings strong ETL/ELT testing experience, SQL proficiency, and hands-on work with Azure data technologies such as Databricks, ADLS, ADF, Delta Lake, Kafka, and DevOps. Collaborate with teams in a fast-paced data-driven environment.