Data Quality Engineer - Databricks, SQL & ETL

ConnectOS; Metro Manila; On-site; PHP 600,000 - 1,000,000; Full time

Benefits offered by this job

Medical Coverage

Vacation Leave

Salary & Appraisal

Financial Assistance

13th Month Pay

Office Fitness

Wellness Facilities

Company Events

Central Location

Job summary

ConnectOS is hiring a Data QA Engineer for our hybrid office in Mega Tower, Mandaluyong, Metro Manila. You will validate data across multiple platforms, design tests for ingestion, transformation, and data products, and lead source-to-target reconciliation using SQL.

Collaborate with Data Engineers and QA teams to ensure data quality and reliable releases. The role requires 3+ years in data QA, strong SQL, Python, and Databricks skills, plus experience with CI/CD and large datasets.

Qualifications

• 3+ years of experience in Data QA, Data Quality Engineering role
• Strong SQL skills for data validation, reconciliation, investigation, and root-cause analysis
• Hands-on experience with Databricks notebooks, workflows, jobs, monitoring, and troubleshooting
• Experience using Python for data validation and test automation within Databricks notebooks
• Strong experience testing data pipelines, ETL/ELT processes, transformations, and data products
• Experience validating data ingestion, transformation, enrichment, and consumption processes
• Experience performing source-to-target reconciliation and investigating data discrepancies
• Understanding of batch and event-driven data pipelines and distributed data architectures
• Experience monitoring and troubleshooting Databricks jobs and workflows
• Experience working with large and complex datasets
• Familiarity with CI/CD pipelines and modern deployment practices
• Knowledge of software testing principles, methodologies, and defect management
• Strong analytical and problem-solving skills with excellent attention to detail
• Strong English communication skills, with the ability to clearly communicate data quality issues, findings, and technical concerns to both technical and non-technical stakeholders
• Experience with Azure, PySpark, Event Hubs/Kafka, or data observability tools is an advantage

Responsibilities

• Validate data accuracy, completeness, consistency, and integrity across data products and platforms
• Design and execute tests for data ingestion, transformation, enrichment, and consumption
• Perform source-to-target reconciliation and data validation using SQL
• Validate business rules, calculations, and transformation logic
• Test batch and event-driven data pipelines throughout the development lifecycle
• Perform integration, system, regression, and end-to-end testing of data solutions
• Monitor Databricks jobs, workflows, and platform processes
• Investigate and troubleshoot Databricks job and data pipeline failures
• Analyze execution logs, alerts, and monitoring information to identify data quality issues
• Develop and maintain automated data validation and reconciliation solutions using SQL, Python, and Databricks notebooks
• Contribute to automated testing within CI/CD pipelines
• Identify, document, investigate, and support the resolution of data defects and quality issues
• Work closely with Data Engineers, Developers, Architects, Product Owners, and QA teams
• Support data migrations, platform upgrades, and release readiness activities
• Contribute to improving data testing, automation, monitoring, and quality assurance practices

Skills

Data QA experience

SQL for data validation

Databricks notebooks/workflows

Python for data validation

Data pipelines testing

Ingestion/transformation validation

Source-to-target reconciliation

Batch/event-driven pipelines

CI/CD familiarity

Software testing principles

English communication

Analytical/problem-solving

Tools

Databricks

Azure

Event Hubs/Kafka

PySpark

Job description

ConnectOS is hiring a Data QA Engineer for our hybrid office in Mega Tower, Mandaluyong, Metro Manila. You will validate data across multiple platforms, design tests for ingestion, transformation, and data products, and lead source-to-target reconciliation using SQL.

Collaborate with Data Engineers and QA teams to ensure data quality and reliable releases. The role requires 3+ years in data QA, strong SQL, Python, and Databricks skills, plus experience with CI/CD and large datasets.