Data Testing Automation Engineer (Python, PySpark, Databricks)

Quesscorp Singapore Pte Ltd; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

Quesscorp Singapore Pte Ltd is seeking a Mid-Level QA / Testing Automation Engineer with strong data testing experience in Python, SQL, PySpark, and Big Data environments. You will develop automated testing frameworks for data validation and support migration testing across data pipelines.

The role emphasizes transitioning from manual SQL checks to Python-based automation and may favor Databricks knowledge and related tooling.

Qualifications

• 5+ years of QA/Testing experience.
• 3+ years in Data Testing in Big Data, Hadoop, or Cloud Data Warehouse environments.
• Proven experience transitioning from manual SQL checks to Python automation.
• Experience with Databricks preferred; certifications a plus.

Responsibilities

• Develop and maintain automated Python-based testing frameworks for data validation.
• Perform data testing, reconciliation, and migration testing using Python and SQL.
• Develop automated tests for PySpark transformations and data pipelines.
• Perform regression, integration, scalability, and end-to-end pipeline testing.
• Validate data across Bronze, Silver, and Gold layers.
• Test Databricks workflows, schema evolution, data lineage, and data quality.
• Identify defects and work with Data Engineers on root-cause analysis.

Skills

QA testing

Data validation

Python

SQL

PySpark

Tools

Databricks

Great Expectations

DLT Expectations

pytest

Job description

Quesscorp Singapore Pte Ltd is seeking a Mid-Level QA / Testing Automation Engineer with strong data testing experience in Python, SQL, PySpark, and Big Data environments. You will develop automated testing frameworks for data validation and support migration testing across data pipelines.

The role emphasizes transitioning from manual SQL checks to Python-based automation and may favor Databricks knowledge and related tooling.