ETL Testing with Bigquery

enGen Global; Hyderabad, Chennai; Sur place; INR 1 200 000 - 1 600 000; Plein temps

Résumé du poste

enGen Global is seeking an experienced ETL Senior Tester to validate ETL/ELT data pipelines and ensure data integrity for analytics and reporting. You will design comprehensive test plans, execute manual and automated tests, and collaborate with data engineers, analysts, and stakeholders to ensure data quality and reliability.

The role requires strong SQL, PySpark scripting, and experience with cloud platforms like GCP, Azure, or AWS, within an Agile environment.

Qualifications

• Bachelor's degree or higher from a reputed university.
• 6–10 years of manual and automated testing, focused on ETL and data validation.
• Strong understanding of ETL processes and data warehousing concepts.
• Advanced SQL skills for data validation, reconciliation, and integrity checks.
• Hands-on PySpark scripting for test automation.
• Experience with cloud platforms such as Google Cloud Platform, Azure, AWS.

Responsabilités

• Test Strategy & Planning: design test plans, scenarios, and cases for ETL workflows.
• ETL Testing Execution: perform manual and automated testing of ETL processes.
• Data Validation & Integrity: validate mappings across source, staging, data warehouse, and reporting layers.
• SQL Proficiency: write complex queries for data validation and integrity checks.
• Defect Management: log and track defects using Jira or Azure DevOps.
• Performance Testing: analyze pipeline performance, throughput, and latency.
• Documentation & Reporting: document results, defects, evidence, and sign-off artifacts.
• Agile Collaboration: participate in Agile ceremonies and cross-functional teamwork.
• Continuous Improvement: contribute to QA process improvements over time.

Connaissances

SQL proficiency

ETL testing

Data validation

PySpark scripting

Agile collaboration

Data warehousing concepts

Analytical thinking

Communication skills

Formation

Bachelor's degree or higher

Outils

Jira

Azure DevOps

PySpark

Dataproc

Databricks

Google Cloud Platform

AWS

Description du poste

Role Summary:

We are seeking a highly skilled and detail-oriented ETL Senior Tester to join our growing data team. The successful candidate will be responsible for validating the accuracy, completeness, and integrity of data as it flows through our enterprise-grade Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes and data pipelines. This role is crucial for ensuring the reliability of our data for analytics, reporting, and critical business operations. The ETL Tester will play a key role in designing and executing comprehensive test plans, scenarios, and cases for complex data transformations, data migrations, and data warehousing solutions. They will collaborate closely with data engineers, developers, business analysts, and other stakeholders to ensure that our data solutions meet business requirements

Essential Responsibilities:

• Test Strategy & Planning: Analyze business, functional, and technical requirements to identify testable scenarios and clarify ambiguities in user stories and specifications. Design comprehensive test plans, test scenarios, and test cases for ETL workflows, data platforms, and reporting solutions.
• ETL Testing Execution: Execute manual and automated testing for ETL processes, data transformations, data validation, and data pipelines.
• Data Validation & Integrity: Perform thorough data validation, data comparison, and backend testing to ensure the accuracy of data transformations and mappings. Validate data mapping, transformations, aggregations, and business rules across source systems, staging layers, data warehouses, and reporting layers. Conduct data integrity checks, reconciliations, and consistency validations using advanced SQL and other data validation techniques.
• SQL Proficiency: Write and execute complex SQL queries for data validation, reconciliation, integrity checks, and to verify data transformations and business rules.
• Defect Management: Identify, log, track, and manage defects using tools such as Jira or Azure DevOps, and collaborate with developers and data engineers to resolve issues and verify fixes.
• Performance Testing: Support ETL performance testing and validate load performance against benchmarks. Analyze pipeline runtime, throughput, latency, and resource utilization.
• Documentation & Reporting: Document test execution results, defect reports, test evidence, and sign-off artifacts to support quality assurance, audits, and compliance. Communicate test results, risks, quality metrics, and recommendations clearly to cross-functional stakeholders.
• Agile Collaboration: Actively participate in Agile delivery processes, including Program Increment planning, stand-ups, sprint planning, grooming, and retrospectives, and collaborate effectively with cross-functional teams.
• Continuous Improvement: Incorporate learnings and best practices to improve the ETL testing process over time and contribute to continuous QA process improvements

The experience we are looking to add to our team

Required

• Bachelors degree or higher from a reputed university
• 6 to 10 years of experience in manual and automated testing, with a strong focus on ETL, data transformation, and data validation.
• Strong understanding of ETL processes, data warehousing concepts (facts, dimensions, star/snowflake schema), relational databases.
• Advanced SQL skills, including writing complex queries for data validation, reconciliation, and integrity checks (joins, aggregations, subqueries, window functions).
• Hands-on scripting experience using PySpark for test automation.
• Hands-on experience creating test cases, test scenarios, and executing test plans
• Experience with cloud platforms such as Google Cloud Platform or Azure, AWS
• Experience with defect tracking tools such as Jira or Agility etc.
• Strong analytical, problem-solving, and communication skills
• Ability to work independently and collaboratively in an Agile environment

Good to have

• Dataproc or Databricks experience
• Healthcare industry experience
• Familiarity with healthcare data models (claims, clinical, eligibility, provider data, etc.)
• Understanding of data governance and data quality standards