ETL Tester with Python and Databricks Experience

InfoBeans; Indore District, Pune District; On-site; INR 700,000 - 1,000,000; Full time

Job summary

Infobeans is seeking an ETL QA Tester with Python and Databricks experience to ensure data integrity across the Databricks platform. You will validate complex data pipelines using SQL and Python, collaborating with global teams to deliver high-quality data solutions.

This role requires flexibility to align with US EST working hours for real-time coordination and team collaboration.

Qualifications

• Advanced SQL: proficiency with window functions, complex joins & CTEs.
• ETL and Data Warehouse concepts understood.

Responsibilities

• Design, develop, and execute ETL test cases to validate data extraction, transformation and loading processes.
• Write and execute complex SQL queries to compare data across hops for validation.
• Medallion Architecture Testing: validate Bronze, Silver, Gold data layers in Databricks.
• Write and run advanced SQL and Python scripts to validate large datasets and identify defects.
• Design, develop, and execute UI Front end manual test cases to verify software functionality and track defects.
• Participate in project meetings and be available for discussions in US EST working hours as needed.

Skills

Advanced SQL

ETL & DWH

Databricks experience

Python (PySpark / Pandas)

Azure / AWS

Agile / Scrum

Tools

Databricks Notebooks

Jira

Git

Job description

Job Title: ETL QA Tester with Python and Databricks experience

We are seeking a meticulous ETL QA Engineer to ensure data integrity across our Databricks platform.

You will validate complex data pipelines using SQL and Python, working closely with global teams to deliver high-quality data solutions.

This role requires flexibility to ensure overlap with US EST working hours for real-time coordination and team collaboration.

Key Responsibilities

• Design, develop, and execute ETL test cases to validate data extraction, transformation, and loading processes.
  Write and execute complex SQL queries to perform data validation between multiple hops.
• Medallion Architecture Testing: Validate data integrity as it moves through the Bronze, Silver, and Gold layers within Databricks.
• Write and execute advanced SQL and Python scripts to validate large datasets and identify defects.
• Design, develop, and execute UI Front end manual test cases to validate the functionalities of the software.
  Identify, document, and track defects, collaborate with developers to ensure timely resolution.
• Participate in project meetings and be available for discussions in US EST working hours as needed

Required Skills & Qualifications

Must-have

• Advanced SQL: Proficiency in windows functions, complex joins and CTEs for deep data validation.
• ETL & DWH: Deep understanding of ETL and Data Warehouse concepts.

Good to have

• Databricks Experience: Hands-on experience with Databricks Notebooks and cluster management
• Programming: Strong skills in Python, specifically PySpark and Pandas for large-scale data manipulation.
• Cloud Platform Familiarity: Experience working within Azure or AWS environments.
• Agile Methodologies and Defect Tracking Tools: Experience with Jira for defect tracking and working with Agile/Scrum framework.
• CI/CD (Additional Plus): Familiarity with automated deployment pipelines and version control (Git).