Data Quality Engineer (Data QE)

AlgoLeap Technologies Pvt Ltd.; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Algoleap Technologies Pvt Ltd in Hyderabad seeks a Data Quality Engineer to ensure data quality across enterprise platforms, validating pipelines from CDP, GA4, BigQuery, and Blob Storage into Snowflake, with Power BI reporting validation. Expertise in ETL/ELT, data governance, and data lineage is required.

The candidate will design, execute, and automate tests using SQL, Python, and PySpark, and integrate tests into CI/CD pipelines using Azure DevOps or GitHub Actions.

Qualifications

• Experience with data validation across enterprise data platforms.
• Experience validating data pipelines from source systems to Snowflake.

Responsibilities

• Perform end-to-end data validation across ingestion, transformation, storage and reporting.
• Validate source-to-target data across CDP, GA4, BigQuery, Blob Storage, PostgreSQL, MSSQL into Snowflake.
• Validate Power BI reports, dashboards, and KPIs.
• Automate data quality checks and integrate tests into CI/CD pipelines.

Skills

Data QA

Data Warehousing

Medallion Architecture

ETL/ELT testing

Data Governance

Data Quality frameworks

Data Lineage

Test automation

SQL

Python

PySpark

CI/CD

Azure DevOps

GitHub Actions

Power BI

Tools

Snowflake

BigQuery

PostgreSQL

Microsoft SQL Server

Power BI

Soda

Azure DevOps

GitHub Actions

Job description

Algoleap Technologies Pvt Ltd | Full time

We are seeking an experienced Data Quality Engineer (DataQE) to ensure the quality, accuracy, reliability, and integrity of dataacross enterprise data platforms. The ideal candidate will have strongexperience in Data Warehousing, Medallion Architecture, ETL/ELT testing,Data Governance, Data Quality frameworks, Data Lineage, and test automation.This role will be responsible for validating data pipelines from multiplesource systems such as CDP, GA4, BigQuery, and Azure Blob Storage into Snowflake,and ensuring accurate reporting in Power BI.

Required Experience

• __years of experience in Data Quality Engineering, Data Warehouse Testing,or Data Validation.
  
• Stronghands-on experience with PostgreSQL, Microsoft SQL Server, Snowflake andBigQuery.
  
• Experiencevalidating enterprise Data Warehouse solutions and Medallion Architectureimplementations.
  
• Provenexperience performing Source-to-Target validation from CDP, GA4, BigQuery,Blob Storage, PostgreSQL, and MSSQL into Snowflake.
  
• Experiencein validating Power BI reports, dashboards, semantic models, and KPIs.
  
• Strongunderstanding of ETL/ELT data processing, data transformations, and datalineage.
  
• Experienceworking with Data Governance, Data Quality, Metadata Management, DataLineage, Data Dictionary, and Data Catalog solutions.
  
• Experiencewith Data Quality tools such as Soda or equivalent.
  
• Experiencedesigning, executing, and automating test cases for large-scale dataplatforms using SQL, Python, and PySpark.
  
• Experienceintegrating automated testing into CI/CD pipelines using Azure DevOps orGitHub Actions.
  

Key Responsibilities

Data Validation & Testing

• Performend-to-end data validation across the data lifecycle, including ingestion,transformation, storage, and reporting.
  
• Validatedata movement from source systems including:
  
• CustomerData Platforms (CDP)
  
• GoogleAnalytics 4 (GA4)
  
• GoogleBigQuery
  
• MSSQL
  
• Snowflake
  
• Performcomprehensive Source-to-Target (S2T) data validation between sourcesystems and Snowflake.
  
• Validatedata accuracy, completeness, consistency, uniqueness, and timelinessacross datasets.
  
• Executedata reconciliation and data profiling activities.
  
• Conductsource-to-target validation across databases, data lakes, and datawarehouses.
  
• Validateschema changes, constraints, indexes, stored procedures, functions,triggers, and views.
  

Medallion Architecture Validation

• Validatedata across Bronze, Silver, and Gold layers within the MedallionArchitecture.
  
• Verifydata transformations, cleansing, aggregates, and business rules betweenlayers.
  
• Ensuredata quality controls are enforced throughout the Medallion framework.
  

ETL/ELT Testing

• Testand validate ETL/ELT pipelines and data transformations.
  
• Validatebusiness mappings, transformation logic, and data lineage.
  
• Verifyincremental and full-load processes.
  
• Performregression, integration, functional, and end-to-end testing of datapipelines.
  
• Identifyand troubleshoot data discrepancies across source and target systems.
  

Snowflake Data Validation

• Validatedata ingestion, transformations, and storage within Snowflake.
  
• Performlarge-scale data validation using SQL and automation scripts.
  
• ValidateSnowflake views, tables, materialized views, stored procedures, and datasharing mechanisms.
  

Power BI Testing

• Validatedatasets, data models, measures, KPIs, and dashboards.
  
• EnsurePower BI reports accurately reflect Snowflake data.
  
• Verifyreport-level calculations, filters, row-level security, and businessmetrics.
  
• Performend-to-end testing from source systems through Snowflake into Power BI.
  

Data Governance & Data Management

• Validateimplementation of Data Governance policies and standards.
  
• Ensurecompliance with enterprise data quality and governance requirements.
  
• Validateand maintain:
  
• DataDictionary
  
• MetadataManagement
  
• BusinessGlossary
  
• DataCatalogs
  
• Collaboratewith Data Governance and Business teams to improve data quality processes.
  

Data Quality Frameworks

• Implementand execute data quality checks and monitoring solutions.
  
• Defineand monitor Data Quality KPIs and metrics.
  
• Developand automate validation rules for:
  
• Accuracy
  
• Consistency
  
• Validity
  
• Timeliness
  
• Supportroot cause analysis and remediation of data quality issues.
  

Test Automation

• Developautomated data validation frameworks and reusable test suites.
  
• Automatesource-to-target reconciliation and regression testing.
  
• Integrateautomated data quality tests into CI/CD pipelines.
  
• Buildautomated validation using SQL, Python, PySpark, and Data Quality tools.
  

Required Technical Skills

Databases & Data Warehousing

• MicrosoftSQL Server (MSSQL)
  
• Snowflake
  
• BigQuery
  
• DataWarehouse Testing
  
• DataModeling Concepts
  
• StarSchema & Snowflake Schema
  
• SlowlyChanging Dimensions (SCD)
  
• Fact& Dimension Validation
  
• CDCValidation
  
• DataMigration Testing
  

SQL Expertise

• AdvancedSQL Query Writing
  
• ComplexJoins
  
• WindowFunctions
  
• Aggregations
  
• QueryOptimization
  
• DataProfiling and Data Analysis
  

Modern Data Platform Validation

• MedallionArchitecture Validation (Bronze, Silver, Gold)