InfoBeans - ETL Tester - Databricks/Python

InfoBeans; Indore District; On-site; INR 900,000 - 1,300,000; Full time

Job summary

InfoBeans is seeking a meticulous ETL QA Engineer to ensure data integrity across our Databricks platform. You will validate data pipelines using SQL and Python, collaborating with global teams to deliver high-quality data solutions. Flexibility to overlap with US EST hours is required.

Responsibilities include designing and executing ETL test cases, validating Bronze-Silver-Gold layers, and performing UI front-end testing. You will track defects and participate in project discussions.

Qualifications

• Experience validating ETL/data pipelines using SQL and Python.
• Strong SQL including window functions, joins, and CTEs.
• Experience with Databricks notebooks and cluster management.
• Knowledge of Data Warehouse concepts and DWH testing.

Responsibilities

• Design, develop, and execute ETL test cases for data pipelines.
• Write complex SQL queries for data validation.
• Validate Bronze/Silver/Gold via Medallion Architecture testing.
• Write Python scripts to validate large datasets.
• Perform UI front-end manual test cases.
• Identify and track defects with timely resolution.
• Participate in US EST meetings as needed.

Skills

Advanced SQL

ETL testing

Data validation

Python

UI testing

Tools

Databricks

Jira

Git

Job description

Summary

We are seeking a meticulous ETL QA Engineer to ensure data integrity across our Databricks platform. You will validate complex data pipelines using SQL and Python, working closely with global teams to deliver high-quality data solutions. This role requires flexibility to ensure overlap with US EST working hours for real-time coordination and team collaboration.

Key Responsibilities

• Design, develop, and execute ETL test cases to validate data extraction, transformation, and loading processes.


• Write and execute complex SQL queries to perform data validation between multiple hops.


• Medallion Architecture Testing: Validate data integrity as it moves through the Bronze, Silver, and Gold layers within Databricks.


• Write and execute advanced SQL and Python scripts to validate large datasets and identify defects.


• Design, develop, and execute UI Front end manual test cases to validate the functionalities of the software.


• Identify, document, and track defects, collaborate with developers to ensure timely resolution.


• Participate in project meetings and be available for discussions in US EST working hours as needed.

Required Skills & Advanced SQL: Proficiency in windows functions, complex joins and CTEs for deep data validation.

• ETL & DWH: Deep understanding of ETL and Data Warehouse concepts.

Good To Have

• Databricks Experience: Hands-on experience with Databricks Notebooks and cluster management.


• Programming: Strong skills in Python, specifically PySpark and Pandas for large-scale data manipulation.


• Cloud Platform Familiarity: Experience working within Azure or AWS environments.


• Agile Methodologies and Defect Tracking Tools: Experience with Jira for defect tracking and working with Agile/Scrum framework.


• CI/CD (Additional Plus): Familiarity with automated deployment pipelines and version control (Git).

(ref:hirist.tech)