IT Software Quality Engineer (ETL / Big Data)

United States Digital Space LLC; Rotterdam; On-site; EUR 55,000 - 85,000; Full time

Job summary

United States Digital Space LLC is seeking an IT Software Quality Engineer to drive QA across complex data platforms and cloud ecosystems. The role focuses on ETL and Big Data testing, data validation, SQL, and test automation to ensure accuracy and reliability of end-to-end data pipelines.

You will plan and execute functional, integration, system, and regression tests, build automated validation frameworks in Python/PySpark, and validate data across Azure Data Factory, Databricks, and Data Lake

Qualifications

• Must-have: ETL Testing / Big Data Testing, Manual Testing, BI/Reporting Validation, Advanced SQL, Data Validation and Reconciliation, End-to-End Test Lifecycle Management, Azure DevOps, ETL Frameworks and Processes.
• Hands-on experience: Python, PySpark, Pandas, Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Gen2, Postman.
• Bachelor's or Master's Degree in Computer Science, Information Technology, Data Engineering, or related technical discipline.
• Strong analytical and problem-solving skills are required.

Responsibilities

• Perform end-to-end testing of ETL and data workflows.
• Validate data extraction, transformation, and loading processes.
• Conduct data reconciliation and cross-system validation.
• Create test plans and execute functional, integration, system, and regression tests.
• Develop automated data validation frameworks using Python, PySpark, and Pandas.
• Test cloud data pipelines on Azure Data Factory, Databricks, and Data Lake Gen2.
• Validate data exchange across integrated applications and perform API testing with Postman.
• Manage defect lifecycle and report testing metrics to stakeholders.

Skills

ETL Testing

Big Data Testing

Manual Testing

BI/Reporting Validation

Advanced SQL

Data Validation & Reconciliation

End-to-End Test Lifecycle

Azure DevOps

ETL Frameworks

Python

PySpark

Pandas

Postman

Azure Data Factory

Azure Databricks

Azure Data Lake Gen2

Education

Bachelor's degree in Computer Science

Master's degree in Data Engineering

Tools

Postman

Azure Data Factory

Azure Databricks

Azure Data Lake Gen2

Python

SQL

Job description

IT Software Quality Engineer (ETL / Big Data)

We are working with a leading company seeking an experienced IT Software Quality Engineer to drive quality assurance across complex data platforms and cloud-based ecosystems. The role requires strong expertise in ETL and Big Data testing, data validation, SQL, and test automation, with a focus on ensuring data accuracy, integrity, and reliability across end-to-end data pipelines.

Key Responsibilities

ETL & Data Validation

• Perform end-to-end testing of ETL and data workflows
• Validate data extraction, transformation, and loading processes
• Conduct data reconciliation and cross-system validation
• Ensure data quality, completeness, consistency, and accuracy across source and target systems

Test Planning & Execution

• Analyze business requirements and create detailed test scenarios and test cases
• Develop comprehensive test strategies aligned with business and technical requirements
• Execute functional, integration, system, and regression testing
• Ensure adequate test coverage and quality across releases

SQL & Data Analysis

• Perform advanced SQL-based data validation and analysis
• Validate complex data transformations, aggregations, joins, and business rules
• Investigate and troubleshoot data-related defects and inconsistencies

Automation & Tools (good to have)

• Develop and maintain automated data validation frameworks using Python, PySpark, and Pandas
• Create reusable automation utilities to improve testing efficiency
• Support continuous improvement of test automation capabilities

Cloud & Data Platform Testing

• Validate data pipelines built on Azure Data Factory (ADF)
• Test data processing and storage solutions on Azure Databricks and Azure Data Lake
• Ensure seamless integration between cloud-based data services

API & Integration Testing

• Validate data exchange across integrated applications and services
• Perform API testing using tools such as Postman

Defect Management

• Manage the complete defect lifecycle, from identification through closure
• Collaborate closely with development and integration teams to resolve issues effectively

Reporting & Stakeholder Communication

• Track and report testing progress, quality metrics, risks, dependencies, and blockers
• Provide regular status updates to project stakeholders
• Work closely with developers, business analysts, product owners, and business users

Quality & Risk Management

• Identify quality risks proactively and drive mitigation plans
• Ensure compliance with organizational QA standards, processes, and governance practices

Required Skills & Experience

Must-have: ETL Testing / Big Data Testing, Manual Testing, BI/Reporting Validation, Advanced SQL, Data Validation and Reconciliation, End-to-End Test Lifecycle Management, Azure DevOps, ETL Frameworks and Processes Hands-on experience: Python, PySpark, Pandas, Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Gen2, Postman

Educational Qualification

Bachelor's or Master's Degree in Computer Science, Information Technology, Data Engineering, or a related technical discipline

Key Competencies

Strong analytical and problem-solving skills

Attention to detail and quality focus

Ownership and accountability

Effective stakeholder management

Excellent verbal and written communication skills

Proactive risk identification and mitigation

Preferred Candidate Profile

Strong experience in data-centric testing environments

Exposure to cloud-based data platforms and modern data architectures

Ability to independently lead testing activities and coordinate with stakeholders

Interest in emerging AI-driven testing approaches

#QualityEngineer #ETLTesting #BigData #AzureJobs #SQLJobs #DataValidation #HiringNow #TestAutomation #PySpark #Databricks #TechCareers