ETL Lead

ACL Digital; Bengaluru; Hybrid; INR 4,000,000 - 7,000,000; Full time

Job summary

ACL Digital seeks an experienced data quality assurance professional to lead end‑to‑end QA for large data warehouse migrations. You will leverage SQL, Python, and PySpark to validate data across lakes and warehouses within a hybrid work model in Bengaluru.

The role requires 8–10 years in QA with 4+ years in data warehouse/ETL testing. Strong AWS Databricks familiarity and experience with CI/CD and Jira-based test management are essential.

Qualifications

• 8–10 years in software/data QA with 4+ years in data warehouse/ETL/big data testing.
• Delivery of at least two end-to-end data warehouse migrations with millions of records.
• Expert SQL with complex joins, window functions, and set-based comparisons.
• Strong Python and PySpark for test automation and DataFrame comparisons.
• Knowledge of AWS data services and Databricks stack.
• Experience designing data reconciliation approaches for very large volumes.
• Agile delivery experience with Git, CI/CD, and test management tools.
• Bachelor's degree in CS/IT/Engineering or equivalent experience.

Skills

Data quality assurance

Data warehouse testing

ETL testing

Big data testing

SQL

Python

PySpark

Test automation

Agile delivery

Education

Bachelor's degree in CS/IT/Engineering

Tools

Databricks

Redshift

Teradata

Oracle

Snowflake

AWS Glue

Jira (Xray/Zephyr)

Job description

Notice period : 15 Days to Immediate

Interview Mode : Virtual

Work Mode : Hybrid

Job Description :

Required Qualifications

• 8-10 years in software/data quality assurance, with at least 4 years focused on data warehouse, ETL, or big data testing.
• Hands-on delivery of at least two end-to-end data warehouse migrations involving millions of records or more, ideally Redshift to Databricks (comparable experience such as Teradata, Oracle, Netezza, Synapse, or Snowflake to Databricks will be considered).
• Expert SQL: complex joins, window functions, set-based comparisons (EXCEPT/MINUS), and performance-aware query writing on large datasets.
• Strong Python and PySpark for test automation and DataFrame-level comparison at scale.
• Working knowledge of AWS data services (S3, Redshift, Glue, IAM, Lambda/Step Functions) and Databricks (Delta Lake, Unity Catalog, Workflows, Databricks SQL).
• Proven ability to design reconciliation approaches (hashing, checksums, profiling, sampling) for very large volumes where full row-by-row comparison is impractical.
• Experience with Agile delivery, Git, CI/CD, and test management tools (Jira with Xray/Zephyr, qTest, or similar).
• Bachelor's degree in Computer Science, IT, Engineering, or a related field, or equivalent practical experience.