Data Quality Engineer Data QE

algoleap; Hyderabad; On-site; INR 1,200,000 - 2,200,000; Full time

Job summary

AlgoLeap Technologies Pvt Ltd. is seeking an experienced Data Quality Engineer to ensure data quality, accuracy, and integrity across enterprise data platforms.

The role covers data validation from CDP, GA4, BigQuery, and Blob Storage into Snowflake, with emphasis on Power BI reporting and data governance. The ideal candidate will have hands-on experience with PostgreSQL, MSSQL, Snowflake, and BigQuery, and will design automated tests for data pipelines, applying QI metrics and data lineage

Qualifications

• Experience in Data Quality Engineering, Data Warehouse Testing, or Data Validation.
• Hands-on experience with PostgreSQL, MSSQL, Snowflake and BigQuery.
• Experience validating enterprise Data Warehouse solutions and Medallion Architecture implementations.
• Proven experience performing Source-to-Target validation from CDP, GA4, BigQuery, Blob Storage, PostgreSQL and MSSQL into Snowflake.
• Experience validating Power BI reports, dashboards, semantic models, and KPIs.
• Knowledge of ETL/ELT data processing, data transformations, and data lineage.
• Experience with Data Governance, Data Quality, Metadata Management, Data Catalogs.

Responsibilities

• Perform end-to-end data validation across the data lifecycle, including ingestion, transformation, storage, and reporting.
• Validate data movement from source systems (CDP, GA4, BigQuery, Blob Storage, PostgreSQL, MSSQL) into Snowflake.
• Perform Source-to-Target data validation between source systems and Snowflake.
• Validate data accuracy, completeness, consistency, uniqueness, and timeliness.
• Execute data reconciliation and profiling activities.
• Validate Medallion Architecture across Bronze, Silver, and Gold layers.
• Test and validate ETL/ELT pipelines and data transformations.
• Validate Snowflake data ingestion, transformations, and storage; test Power BI datasets and dashboards.

Skills

Data Warehousing

Medallion Architecture

ETL testing

Data Governance

Data Quality frameworks

Data Lineage

Test automation

Tools

PostgreSQL

Microsoft SQL Server

Snowflake

BigQuery

Job description

Job Description:

Job Summary

We are seeking an experienced Data Quality Engineer (Data QE) to ensure the quality, accuracy, reliability, and integrity of data across enterprise data platforms. The ideal candidate will have strong experience in Data Warehousing, Medallion Architecture, ETL/ELT testing, Data Governance, Data Quality frameworks, Data Lineage, and test automation. This role will be responsible for validating data pipelines from multiple source systems such as CDP, GA4, BigQuery, and Azure Blob Storage into Snowflake, and ensuring accurate reporting in Power BI.

Required Experience

• __ years of experience in Data Quality Engineering, Data Warehouse Testing, or Data Validation.
• Strong hands‑on experience with PostgreSQL, Microsoft SQL Server, Snowflake and BigQuery.
• Experience validating enterprise Data Warehouse solutions and Medallion Architecture implementations.
• Proven experience performing Source-to-Target validation from CDP, GA4, BigQuery, Blob Storage, PostgreSQL, and MSSQL into Snowflake.
• Experience in validating Power BI reports, dashboards, semantic models, and KPIs.
• Strong understanding of ETL/ELT data processing, data transformations, and data lineage.
• Experience working with Data Governance, Data Quality, Metadata Management, Data Lineage, Data Dictionary, and Data Catalog solutions.
• Experience with Data Quality tools such as Soda or equivalent.
• Experience designing, executing, and automating test cases for large‑scale data platforms using SQL, Python, and PySpark.
• Experience integrating automated testing into CI/CD pipelines using Azure DevOps or GitHub Actions.

Key Responsibilities

Data Validation & Testing

• Perform end‑to‑end data validation across the data lifecycle, including ingestion, transformation, storage, and reporting.
• Validate data movement from source systems including:
  • Customer Data Platforms (CDP)
  • Google Analytics 4 (GA4)
  • Google BigQuery
  • Azure Blob Storage
  • PostgreSQL
  • MSSQL
  • Snowflake
• Perform comprehensive Source-to-Target (S2T) data validation between source systems and Snowflake.
• Validate data accuracy, completeness, consistency, uniqueness, and timeliness across datasets.
• Execute data reconciliation and data profiling activities.
• Conduct source-to-target validation across databases, data lakes, and data warehouses.
• Validate schema changes, constraints, indexes, stored procedures, functions, triggers, and views.

Medallion Architecture Validation

• Validate data across Bronze, Silver, and Gold layers within the Medallion Architecture.
• Verify data transformations, cleansing, aggregations, and business rules between layers.
• Ensure data quality controls are enforced throughout the Medallion framework.

ETL/ELT Testing

• Test and validate ETL/ELT pipelines and data transformations.
• Validate business mappings, transformation logic, and data lineage.
• Verify incremental and full-load processes.
• Perform regression, integration, functional, and end‑to‑end testing of data pipelines.
• Identify and troubleshoot data discrepancies across source and target systems.

Snowflake Data Validation

• Validate data ingestion, transformations, and storage within Snowflake.
• Perform large‑scale data validation using SQL and automation scripts.
• Validate Snowflake views, tables, materialized views, stored procedures, and data sharing mechanisms.

Power BI Testing

• Validate datasets, data models, measures, KPIs, and dashboards.
• Ensure Power BI reports accurately reflect Snowflake data.
• Verify report‑level calculations, filters, row‑level security, and business metrics.
• Perform end‑to‑end testing from source systems through Snowflake into Power BI.

Data Governance & Data Management

• Validate implementation of Data Governance policies and standards.
• Ensure compliance with enterprise data quality and governance requirements.
• Validate and maintain:
  • Data Lineage
  • Data Dictionary
  • Metadata Management
  • Business Glossary
  • Data Catalogs
• Collaborate with Data Governance and Business teams to improve data quality processes.

Data Quality Frameworks

• Implement and execute data quality checks and monitoring solutions.
• Define and monitor Data Quality KPIs and metrics.
• Develop and automate validation rules for:
  • Completeness
  • Accuracy
  • Consistency
  • Uniqueness
  • Validity
  • Timeliness
• Support root cause analysis and remediation of data quality issues.

Test Automation

• Develop automated data validation frameworks and reusable test suites.
• Automate source-to-target reconciliation and regression testing.
• Integrate automated data quality tests into CI/CD pipelines.
• Build automated validation using SQL, Python, PySpark, and Data Quality tools.

Required Technical Skills

Databases & Data Warehousing

• PostgreSQL
• Microsoft SQL Server (MSSQL)
• Snowflake
• BigQuery
• Data Warehouse Testing
• Data Modeling Concepts
• Star Schema & Snowflake Schema
• Slowly Changing Dimensions (SCD)
• Fact & Dimension Validation
• CDC Validation
• Data Migration Testing

SQL Expertise

• Advanced SQL Query Writing
• Complex Joins
• Window Functions
• Aggregations
• Query Optimization
• Data Reconciliation Queries
• Data Profiling and Data Analysis

Modern Data Platform Validation

• Medallion Architecture Validation (Bronze, Silver, Gold)
• Source-to-Target Testing
• ETL/ELT Testing
• Data Lake & Data Warehouse Validation
• Power BI Report & Dashboard Validation

Requirements: