Senior ETL & Data Pipeline Engineer

Digital Room (Philippines) , Inc.; Mandaluyong; On-site; PHP 600,000 - 900,000; Full time

Job summary

Digital Room (Philippines) , Inc. is seeking a Data Engineer 2 to build and operate end-to-end pipelines, from ingestion to publishing, with a focus on orchestration, data lineage, and well‑described datasets consumed by analytics tools and AI systems.

The role requires hands-on development of batch, incremental, and near‑real‑time pipelines, implementation of CDC, and production-grade SQL. You will influence storage formats, partitioning, and schema design to ensure downstream correctness.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field, or equivalent practical experience.
• At least 3 years building and operating production data pipelines (not solely analytics or reporting work).
• Strong Python at production quality: modular code, error handling, logging, configuration, and tests, well beyond one‑off scripts.
• Advanced SQL, including window functions, multi-step CTE transformations, and set operations; able to read an execution plan and explain why a query is slow and what to change.
• Hands‑on AWS data services: S3, Glue, Lambda, and Redshift, plus at least one of DMS, Kinesis, EMR, Step Functions, or Athena.
• Demonstrable understanding of columnar storage and columnar analytical databases: file formats, encoding and compression, partitioning and pruning, and their effect on query cost.
• Experience loading and querying non‑transactional stores, including a data lake on object storage; able to explain how lake and warehouse workloads differ.
• Production experience with at least one orchestration framework, and the ability to reason about workflow design, dependencies, idempotency, retries, and backfills rather than only clicking through a UI.
• Data modeling ability: dimensional and/or wide‑table designs, grain and key definition, and slowly changing dimensions.
• Data quality as routine practice: automated tests and reconciliation as part of delivery, not an afterthought.
• Experience with Git branching, pull requests, and CI‑based deployment.
• Clear written and verbal communication. Able to document a dataset so that someone who did not build it can use it correctly, and able to explain a data issue to a non‑technical stakeholder.
• Ability to analyze broad, ambiguous data needs and decompose them into deliverable work.

Responsibilities

• Build and maintain batch, incremental, and near-real-time pipelines that extract from transactional databases, SaaS and internal APIs, file feeds, and event streams, and load to databases, data lakes, and data warehouses.
• Implement change data capture (CDC) on high-volume source tables, handling schema drift, late-arriving and out-of-order records, soft deletes, and replays.
• Write idempotent, restartable jobs: re-running a pipeline for a given partition or window produces the same result without duplicating or losing rows.
• Apply the correct load pattern for the volume and business need, whether full refresh, watermark-based incremental, merge/upsert, or slowly changing dimension (Type 1 and Type 2), and justify the choice.
• Write and tune complex SQL: window functions, multi-CTE transformations, set operations. Read the query plan and address the actual cause of slowness (scan volume, distribution skew, spill to disk, poor join order) rather than adding hardware.
• Apply business rules and integration logic in a testable, documented layer rather than embedded in ad-hoc scripts.
• Designing the orchestration layer, task dependencies, and observability. Responsibilities include implementing backfill mechanisms, managing failure strategies, and creating thorough documentation.
• Modeling data for consumption with clear definitions of grain, keys, and measures, alongside an understanding of columnar storage mechanics like Parquet and ORC. Responsibilities include applying physical design principles for columnar warehouses, managing file layouts in object storage, and working with non-transactional stores while evaluating and defending the appropriate storage solution for various workloads.
• Designing automated lineage capture at table and column levels, ensuring that documentation stays current with orchestration and transformation changes. Responsibilities include maintaining dataset catalogs with essential details, performing impact analysis before schema changes, and implementing data quality gates to prevent erroneous data from being published. Additionally, focuses on supporting auditability and reproducibility while applying least-privilege access and data retention rules.
• Thorough testing before code promotion, including unit tests and validation against source counts, along with ensuring the target environment is ready for initial data loads. Providing production support for pipelines, managing failures with a focus on root causes, participating in an on‑call rotation for incidents, and working within a Git‑based workflow with peer reviews. Additionally, comprehensive documentation is essential to ensure that datasets and pipelines can be operated by others without assistance.
• Translating business questions from various stakeholders into clear data requirements while suggesting alternative solutions when appropriate. Responsibilities include participating in design and code reviews to address concerns regarding maintainability, cost, and data accuracy, as well as mentoring Level 1 engineers in SQL, pipeline patterns, and debugging techniques.
• Provide production support for the pipelines you own: triage and root‑cause failures, communicate impact and ETA to affected consumers, then eliminate the class of failure rather than only the instance.

Skills

Python

SQL

Data modeling

Git & CI

Documentation & communication

Education

Bachelor’s degree in CS/IT or related field

Tools

S3

Glue

Lambda

Redshift

DMS/Kinesis/EMR/Step Functions/Athena

Job description

Digital Room (Philippines) , Inc. is seeking a Data Engineer 2 to build and operate end-to-end pipelines, from ingestion to publishing, with a focus on orchestration, data lineage, and well‑described datasets consumed by analytics tools and AI systems.

The role requires hands-on development of batch, incremental, and near‑real‑time pipelines, implementation of CDC, and production-grade SQL. You will influence storage formats, partitioning, and schema design to ensure downstream correctness.