Senior Data QA Engineer - Python Automations & Databricks

APAR TECHNOLOGIES PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

APAR TECHNOLOGIES PTE. LTD. in Singapore seeks a senior QA/Data Quality Engineer to validate data using Python-based testing, Great Expectations and Pandera.

You will build automated tests for PySpark transformations, validate Delta Live Tables expectations, and ensure data integrity across the lakehouse. You will collaborate on end-to-end data reconciliation, scale testing to meet performance SLAs, and verify governance controls such as Unity Catalog RLS and data lineage.

Qualifications

• 5+ years of QA/testing experience.
• 3+ years in Big Data / Hadoop / Cloud Data Warehouse environments.
• 1+ year Databricks testing pipelines.
• Experience moving from manual SQL checks to automated Python tests.
• Automation and migration automation testing experience.

Responsibilities

• Validate data contracts with data validation frameworks.
• Write Python-based tests for PySpark transformations.
• Test Databricks workflows and job retries.
• Assess schema evolution impacts on downstream tables.
• Validate Unity Catalog RLS and column masking.

Skills

Data testing experience

Automation mindset

Python programming

Big data testing

Tools

Great Expectations

Pandera

DLT Expectations

Pytest-Spark

Unity Catalog testing

Job description

APAR TECHNOLOGIES PTE. LTD. in Singapore seeks a senior QA/Data Quality Engineer to validate data using Python-based testing, Great Expectations and Pandera.

You will build automated tests for PySpark transformations, validate Delta Live Tables expectations, and ensure data integrity across the lakehouse. You will collaborate on end-to-end data reconciliation, scale testing to meet performance SLAs, and verify governance controls such as Unity Catalog RLS and data lineage.