Azure Data Lake / ETL QA tester (5:00 PM - 2:00 AM)

C-Vision It Private Limited; India; Hybrid; INR 1,800,000 - 3,200,000; Full time

Job summary

C-Vision It Private Limited is seeking an experienced Azure Data Lake/ETL QA Tester to validate data ingestion, transformations, and downstream reporting on Azure-based platforms. You will test ADLS ingestion, ADF pipelines, and Databricks transformations while ensuring data quality across Bronze, Silver, and Gold layers.

The role requires hands-on SQL, PySpark, and experience with data reconciliation, negative testing, and regression planning within an Agile environment.

Qualifications

• Bachelor's degree in Computer Science, IT, Engineering or related field.
• Minimum 5 years of ETL, Data Warehouse, Data Lake, or Data Platform testing.
• Strong hands-on SQL and complex data validation queries.
• Hands-on experience with Azure Data Factory and Azure Data Lake Gen2.
• Experience with Azure Databricks, Python or PySpark, and data reconciliation.
• Familiarity with Agile/Scrum and defect/test-management tools.

Responsibilities

• Review requirements, mappings, specs, and design documents.
• Develop detailed test scenarios for ETL, Data Lake, and Data Warehouse.
• Perform end-to-end validation across pipelines and Bronze/Silver/Gold layers.
• Validate ADLS ingestion, ADF pipelines, and Databricks transformations.
• Execute source-to-target reconciliation: counts, completeness, accuracy.
• Log defects in Azure DevOps/Jira/qTest and drive triage and retesting.
• Support production deployment and participate in Agile ceremonies.

Skills

SQL

Azure Data Factory

Azure Data Lake Storage Gen2

Azure Databricks

Python/PySpark

Data Validation

ETL Testing

Data Warehouse Testing

Education

Bachelor's degree in Computer Science/IT/Engineering

Tools

Azure DevOps

Jira

qTest

Power BI

Tricentis Tosca

Job description

Position Summary

We are looking for an experienced Azure Data Lake / ETL QA Tester to support end-to-end testing of cloud-based data platforms and data pipelines built on Microsoft Azure.

The candidate will be responsible for validating data ingestion, transformation, storage, reconciliation, and downstream consumption across Azure Data Factory, Azure Data Lake Storage, Azure Databricks, Azure SQL/Synapse, and reporting platforms.

The ideal candidate should have strong experience in ETL/Data Warehouse testing, SQL, Azure Data Factory, Azure Data Lake, Databricks, Python/PySpark, and source-to-target data validation.

Key Responsibilities

Review business requirements, data mappings, source-to-target specifications, transformation rules, and technical design documents.

Develop detailed test scenarios and test cases for ETL, Data Lake, and Data Warehouse solutions.

Perform end-to-end validation of data pipelines from source systems through ingestion, transformation, curated layers, and downstream reporting.

Validate data ingestion into Azure Data Lake Storage Gen2 (ADLS).

Test Azure Data Factory pipelines, including:

• Pipeline execution
• Triggers and dependencies
• Incremental and full loads
• Error handling and retries
• Restart and recovery scenarios

Validate data transformations performed in Azure Databricks using SQL, Spark SQL, Python, and PySpark.

Perform source-to-target reconciliation and validate:

• Record counts
• Data completeness
• Data accuracy
• Duplicate records
• Null values
• Referential integrity
• Transformation rules

Validate data across Bronze, Silver, and Gold layers in a Medallion Architecture.

Test full loads, incremental loads, historical loads, and CDC-based processing.

Validate SCD Type 1 and Type 2 transformations where applicable.

Validate Delta Lake tables, schemas, partitions, audit fields, timestamps, and historical data.

Perform negative, boundary, exception, integration, regression, and data-volume testing.

Develop reusable SQL, Python, and PySpark validation s to improve test efficiency and automation.

Validate downstream reporting and analytics data, including Power BI where applicable.

Log, track, and manage defects using Azure DevOps, Jira, qTest, or equivalent tools.

Participate in defect triage, root-cause analysis, retesting, and regression testing.

Maintain test evidence, traceability, test execution results, and quality metrics.

Participate in Agile ceremonies, sprint planning, release-readiness reviews, and production deployment support.

Work closely with Data Engineers, Architects, Business Analysts, Product Owners, Developers, and QA teams.

Required Qualifications

Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.

Minimum 5 years of experience in ETL, Data Warehouse, Data Lake, or Data Platform testing.

Strong hands-on experience with SQL and complex data validation queries.

Hands-on experience with Azure Data Factory (ADF).

Experience testing data stored in Azure Data Lake Storage Gen2.

Experience with Azure Databricks.

Working knowledge of Python and/or PySpark.

Strong experience with source-to-target data reconciliation.

Understanding of ETL/ELT concepts, Data Warehouse, Data Lake, and Lakehouse architectures.

Experience validating Bronze, Silver, and Gold data layers.

Experience with data-quality validation including completeness, accuracy, consistency, duplicates, nulls, and referential integrity.

Experience with Agile/Scrum delivery methodologies.

Experience with defect and test-management tools such as Azure DevOps, Jira, or qTest.

Preferred Qualifications

Experience with Delta Lake and Unity Catalog.

Experience with Azure Synapse Analytics.

Experience with Snowflake or similar cloud data warehouses.

Experience validating Power BI reports and downstream analytics.

Experience with automated data-testing frameworks.

Experience with CI/CD pipelines, Git, GitHub, or Azure DevOps pipelines.

Exposure to Tricentis Tosca or other enterprise test-automation tools.

Experience with metadata-driven pipelines, control tables, audit frameworks, and large-volume data reconciliation.

Key Technical Skills

Mandatory:

• Azure Data Factory, Azure Data Lake Storage Gen2, Azure Databricks, SQL/T-SQL, ETL Testing, Data Warehouse Testing, Source-to-Target Validation, Data Reconciliation, Python/PySpark, Data Quality Testing.

Preferred:

• Delta Lake, Unity Catalog, Azure Synapse, Snowflake, Power BI, CI/CD, Azure DevOps, qTest, Tosca.

Key Competencies

Strong analytical and problem-solving skills

Ability to analyze complex data transformations and business rules

Strong attention to data quality and accuracy

Ability to work independently and within cross-functional teams

Strong communication and defect-management skills

Ability to manage multiple testing priorities in an Agile environment

Success Criteria

The candidate will be expected to ensure that data delivered through the Azure platform is complete, accurate, traceable, reconciled, and fit for downstream business and reporting consumption, while continuously improving test coverage and automation.