ETL Databricks Testing Engineer

Zone IT Solutions; Hyderabad; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

Zone IT Solutions in Hyderabad, India, is seeking an experienced ETL Databricks Testing Engineer to validate end-to-end data pipelines, ensure data quality, and perform data reconciliation across enterprise data platforms. The role requires hands-on Databricks, SQL, and cloud-based data environments.

You will collaborate with Data Engineering, Analytics, and QA teams throughout release cycles, develop test cases, and maintain validation documentation, with focus on Delta Lake, Bronze/Silver/Gold

Qualifications

• Strong experience in ETL Testing and Data Warehouse Testing.
• Hands-on experience with Databricks, including notebooks, clusters, jobs, and Delta Lake.
• Strong SQL skills with expertise in joins, subqueries, aggregations, window functions, and large-scale data validation.
• Good understanding of Data Warehouse concepts, including Facts, Dimensions, Star Schema, and Snowflake Schema.
• Basic to intermediate knowledge of Python for data validation, automation, and log analysis.
• Experience with Spark and PySpark.
• Familiarity with cloud platforms such as Azure, AWS, or GCP.

Responsibilities

• Validate end-to-end ETL and data pipeline workflows from source systems to target data warehouse layers.
• Perform data reconciliation, transformation validation, and data quality checks across Bronze, Silver, and Gold layers.
• Execute Databricks-based data validations using SQL, notebooks, and Delta Lake.
• Develop and execute complex SQL queries for data profiling, completeness, accuracy, and referential integrity validation.
• Validate Delta tables, fact and dimension tables, aggregates, and historical data loads.
• Perform job failure analysis, root cause analysis (RCA), and support defect resolution.
• Collaborate with Data Engineering, Analytics, and QA teams during release cycles.
• Ensure comprehensive test coverage for incremental loads, Change Data Capture (CDC), batch processing, and reprocessing scenarios.
• Create and maintain test cases, test data, and validation documentation.

Skills

ETL Testing

Data Warehouse Testing

Databricks

SQL

Big Data

Python

Spark

PySpark

Azure

AWS

GCP

Tools

Delta Lake

Delta Tables

Job description

We are looking for an experienced ETL Databricks Testing Engineer with strong expertise in ETL/Data Warehouse Testing, Databricks, SQL, and Big Data technologies. The ideal candidate will be responsible for validating end-to-end data pipelines, ensuring data quality, and performing data reconciliation across enterprise data platforms. The role requires hands-on experience in Databricks, SQL, and cloud-based data environments.

Key Responsibilities

• Validate end-to-end ETL and data pipeline workflows from source systems to target data warehouse layers.
• Perform data reconciliation, transformation validation, and data quality checks across Bronze, Silver, and Gold layers.
• Execute Databricks-based data validations using SQL, notebooks, and Delta Lake.
• Develop and execute complex SQL queries for data profiling, completeness, accuracy, and referential integrity validation.
• Validate Delta tables, fact and dimension tables, aggregates, and historical data loads.
• Perform job failure analysis, root cause analysis (RCA), and support defect resolution.
• Collaborate with Data Engineering, Analytics, and QA teams during release cycles.
• Ensure comprehensive test coverage for incremental loads, Change Data Capture (CDC), batch processing, and reprocessing scenarios.
• Create and maintain test cases, test data, and validation documentation.

Required Skills

• Strong experience in ETL Testing and Data Warehouse Testing.
• Hands-on experience with Databricks, including notebooks, clusters, jobs, and Delta Lake.
• Strong SQL skills with expertise in joins, subqueries, aggregations, window functions, and large-scale data validation.
• Good understanding of Data Warehouse concepts, including Facts, Dimensions, Star Schema, and Snowflake Schema.
• Basic to intermediate knowledge of Python for data validation, automation, and log analysis.
• Experience with Spark and PySpark.
• Familiarity with cloud platforms such as Azure, AWS, or GCP.

About Us

We specialize in Digital, ERP, and larger IT Services. We offer flexible, efficient and collaborative solutions to any organization that requires IT, experts. Our agile, agnostic, and flexible solutions will help you source the IT Expertise you need.

Zone IT Solutions is an equal opportunity employer and our recruitment process focuses on essential skills and abilities. We encourage applications from a diverse array of backgrounds, including individuals of various ethnicities, cultures, and linguistic backgrounds, as well as those with disabilities.