ETL Test Automation Engineer – Azure Data Platform

Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!; Mississauga; On-site; CAD 85,000 - 110,000; Full time

Job summary

A technology solutions provider is looking for a QA & Automation Engineer in Mississauga, Canada. This role involves validating ETL pipelines and automating testing processes across Azure data services. The ideal candidate should have 4-10 years of experience in Data Warehouse QA, strong SQL skills, and expertise in ETL tools like Azure Data Factory and Databricks. Responsibilities include data migration testing, performance validation, and developing automated solutions to ensure data quality and integrity.

Qualifications

• 4–10 years of experience in Data Warehouse / Data Engineering QA.
• Strong hands-on expertise with ETL tools such as ADF, SSIS, Informatica, and Databricks.
• Strong analytical, debugging, and problem-solving skills.

Responsibilities

• Perform end-to-end testing of ETL pipelines built using Azure data services.
• Design and execute test strategies for large-scale data migration projects.
• Validate integration flows across Data Lake, Synapse, ADF, and Power BI.
• Develop automated data validation and regression frameworks.

Skills

ETL tools expertise

Expert-level SQL

Data warehousing concepts

Automation frameworks

Tools

Azure Data Factory

SSIS

Jira

Azure DevOps

Databricks

Job description

The QA & Automation Engineer will be responsible for validating ETL pipelines, data migration processes, and cloud data platform integrations across Azure services. This role involves hands‑on testing of large-scale data platforms, automation development, advanced SQL validation, and ensuring end‑to‑end data quality across enterprise data engineering initiatives.

Key Responsibilities

• Perform end‑to‑end testing of ETL pipelines built using Azure Data Factory, Azure Databricks, Azure Synapse, and SSIS.
• Validate data transformations, mappings, data quality rules, and data lineage.
• Conduct source‑to‑target (S2T) reconciliation, data profiling, completeness, and accuracy checks.
• Verify schema changes, incremental loads, delta loads, and historical load processes.

Data Migration Testing

• Design and execute test strategies for large‑scale on‑prem to cloud or cross‑cloud data migration projects.
• Validate ETL/ELT processes and post‑migration data accuracy.
• Perform count checks, checksum validation, CDC validation, duplicate checks, and referential integrity validation.
• Review and validate data mapping documents, business rules, and acceptance criteria.

Data Lake & Cloud Platform Testing

• Test ingestion pipelines into Azure Data Lake Storage (ADLS Gen2) from multiple source systems.
• Validate partitioning, folder structures, file formats (Parquet, CSV, JSON), and governance standards.
• Conduct performance and scalability testing for large data workloads.
• Validate integration flows across Data Lake, Synapse, ADF, and Power BI.

Automation Testing

• Develop automated data validation and regression frameworks using Python, PySpark, and ADF automated validation frameworks.
• Integrate automated tests within CI/CD pipelines using Azure DevOps.
• Build reusable automation templates, accelerators, and validation utilities.
• Create and maintain test plans, scenarios, and test cases in Azure DevOps or Jira.
• Log, track, triage, and validate defects in collaboration with development and data engineering teams.
• Publish daily/weekly QA health metrics, defect reports, coverage summaries, and quality dashboards.

Experience and Qualifications

• 4–10 years of experience in Data Warehouse / Data Engineering QA.
• Strong hands‑on expertise with ETL tools such as ADF, SSIS, Informatica, and Databricks.
• Expert‑level SQL for complex data validation and reconciliation.
• Strong understanding of data warehousing concepts (SCDs, facts/dimensions, star schema).
• Strong analytical, debugging, and problem‑solving skills.

Preferred Skills

• Quality management and defect lifecycle management.
• Integration of automated tests into CI/CD pipelines using Azure DevOps.