End-to-End Data Pipeline Engineer

Digital Room; Mandaluyong; Hybrid; PHP 80,000 - 140,000; Full time

Job summary

Digital Room seeks a Data Engineer 2 to own end-to-end data pipelines from ingestion to publishing. You will design and optimize batch, incremental, and real-time processes, implementing CDC, schema evolution handling, and robust error handling.

You will work with AWS services (S3, Glue, Lambda, Redshift) and several orchestration tools, building clean data models, lineage, and quality gates across data platforms. Collaboration and documentation are essential.

Qualifications

• Bachelor's degree or equivalent practical experience.
• At least 3 years building/operating production data pipelines.
• Strong Python at production quality with modular code, error handling, logging, and tests.
• Advanced SQL with window functions, CTEs, and set operations.
• Hands-on AWS data services (S3, Glue, Lambda, Redshift) and at least one of DMS/Kinesis/EMR/Step Functions/Athena.
• Experience with non-transactional stores, data lakes, and lakehouse concepts.
• Data modeling with grain, keys, and slowly changing dimensions.
• Git, PRs, and CI-based deployment; clear written/verbal communication.

Responsibilities

• Build and maintain batch, incremental, and near-real-time pipelines from diverse sources.
• Implement CDC on high-volume tables and handle schema drift and replays.
• Write idempotent, restartable jobs with consistent results.
• Choose appropriate load patterns and optimize query costs.
• Develop automated lineage, catalogs, and data quality gates.
• Provide production support and on-call incident handling.
• Mentor Level 1 engineers in SQL and pipeline patterns.

Skills

Python

SQL

Data modeling

Git CI

Communication

Education

Bachelor's degree in CS/IT or related field

Tools

AWS S3

AWS Glue

AWS Lambda

Redshift

DMS

Kinesis

EMR

dbt

Dagster

Prefect

Job description

Digital Room seeks a Data Engineer 2 to own end-to-end data pipelines from ingestion to publishing. You will design and optimize batch, incremental, and real-time processes, implementing CDC, schema evolution handling, and robust error handling.

You will work with AWS services (S3, Glue, Lambda, Redshift) and several orchestration tools, building clean data models, lineage, and quality gates across data platforms. Collaboration and documentation are essential.