Data Engineer

Digital Room; Mandaluyong; On-site; PHP 600,000 - 1,200,000; Full time

Job summary

Digital Room is seeking a Data Engineer 2 to build and operate end-to-end data pipelines, from ingestion to publishing. You will own orchestration, data lineage, and dataset modeling to support analytics and AI systems.

A successful candidate will design durable pipelines, justify storage formats and partitioning, and ensure data quality. You will collaborate across teams and contribute to robust data architecture and production readiness.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or related field or equivalent practical experience.
• 3+ years building and operating production data pipelines.
• Strong Python at production quality with modular code, error handling, logging, configuration and tests.
• Advanced SQL with window functions, multi-CTE transformations and set operations; able to read execution plans.

Responsibilities

• Build and maintain batch, incremental, and near-real-time pipelines from diverse sources to data warehouses and lakes.
• Implement CDC on high-volume source tables and handle schema drift and replays.
• Write idempotent, restartable jobs to ensure exactly-once semantics within partitions.
• Apply correct load patterns (full refresh, incremental, upserts) and justify choices.
• Design orchestration layer and observability; manage backfills and failures with robust retry logic.
• Model data with clear grain and keys; document datasets for downstream users.
• Ensure data quality through automated tests and reconciliation as part of delivery.
• Provide production support and participate in on-call rotations; document pipelines for operability.

Skills

Python

SQL

Data modeling

Data quality

Communication

Problem solving

Education

Bachelor’s degree in CS/IT or related field

Equivalent practical experience

Tools

S3

Glue

Lambda

Redshift

DMS

Kinesis

EMR

Step Functions

Athena

dbt

Job description

The Data Engineer 2 builds and operates the pipelines and datasets that the business runs on. This is a hands-on delivery role: the engineer owns pipelines end to end (ingestion, transformation, publishing, monitoring, and support) and is expected to contribute to design decisions above the level of an individual job, particularly in orchestration, data lineage, and how curated data is modeled and exposed to downstream consumers.

Beyond conventional ETL, this role builds the curated, well-described datasets that analytics tools and AI systems consume. A successful candidate can explain not only how to move data, but why a given storage format, partition layout, or table design is the right one, and what breaks downstream when it is wrong.

Responsibilities

• Build and maintain batch, incremental, and near-real-time pipelines that extract from transactional databases, SaaS and internal APIs, file feeds, and event streams, and load to databases, data lakes, and data warehouses.
• Implement change data capture (CDC) on high-volume source tables, handling schema drift, late-arriving and out-of-order records, soft deletes, and replays.
• Write idempotent, restartable jobs: re-running a pipeline for a given partition or window produces the same result without duplicating or losing rows.
• Apply the correct load pattern for the volume and business need, whether full refresh, watermark-based incremental, merge/upsert, or slowly changing dimension (Type 1 and Type 2), and justify the choice.
• Write and tune complex SQL: window functions, multi-CTE transformations, set operations. Read the query plan and address the actual cause of slowness (scan volume, distribution skew, spill to disk, poor join order) rather than adding hardware.
• Apply business rules and integration logic in a testable, documented layer rather than embedded in ad-hoc scripts.
• Designing the orchestration layer, task dependencies, and observability. Responsibilities include implementing backfill mechanisms, managing failure strategies, and creating thorough documentation.
• Modeling data for consumption with clear definitions of grain, keys, and measures, alongside an understanding of columnar storage mechanics like Parquet and ORC. Responsibilities include applying physical design principles for columnar warehouses, managing file layouts in object storage, and working with non-transactional stores while evaluating and defending the appropriate storage solution for various workloads.
• Designing automated lineage capture at table and column levels, ensuring that documentation stays current with orchestration and transformation changes. Responsibilities include maintaining dataset catalogs with essential details, performing impact analysis before schema changes, and implementing data quality gates to prevent erroneous data from being published. Additionally, focuses on supporting auditability and reproducibility while applying least-privilege access and data retention rules.
• Thorough testing before code promotion, including unit tests and validation against source counts, along with ensuring the target environment is ready for initial data loads. Providing production support for pipelines, managing failures with a focus on root causes, participating in an on-call rotation for incidents, and working within a Git-based workflow with peer reviews. Additionally, comprehensive documentation is essential to ensure that datasets and pipelines can be operated by others without assistance.
• Translating business questions from various stakeholders into clear data requirements while suggesting alternative solutions when appropriate. Responsibilities include participating in design and code reviews to address concerns regarding maintainability, cost, and data accuracy, as well as mentoring Level 1 engineers in SQL, pipeline patterns, and debugging techniques.
• Provide production support for the pipelines you own: triage and root-cause failures, communicate impact and ETA to affected consumers, then eliminate the class of failure rather than only the instance.

Required Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field, or equivalent practical experience.
• At least 3 years building and operating production data pipelines (not solely analytics or reporting work).
• Strong Python at production quality: modular code, error handling, logging, configuration, and tests, well beyond one-off scripts.
• Advanced SQL, including window functions, multi-step CTE transformations, and set operations; able to read an execution plan and explain why a query is slow and what to change.
• Hands-on AWS data services: S3, Glue, Lambda, and Redshift, plus at least one of DMS, Kinesis, EMR, Step Functions, or Athena.
• Demonstrable understanding of columnar storage and columnar analytical databases: file formats, encoding and compression, partitioning and pruning, and their effect on query cost.
• Experience loading and querying non-transactional stores, including a data lake on object storage; able to explain how lake and warehouse workloads differ.
• Production experience with at least one orchestration framework, and the ability to reason about workflow design, dependencies, idempotency, retries, and backfills rather than only clicking through a UI.
• Data modeling ability: dimensional and/or wide-table designs, grain and key definition, and slowly changing dimensions.
• Data quality as routine practice: automated tests and reconciliation as part of delivery, not an afterthought.
• Experience with Git branching, pull requests, and CI-based deployment.
• Clear written and verbal communication. Able to document a dataset so that someone who did not build it can use it correctly, and able to explain a data issue to a non-technical stakeholder.
• Ability to analyze broad, ambiguous data needs and decompose them into deliverable work.

Preferred Qualifications

• Open table formats (Apache Iceberg, Delta Lake, Hudi) and lakehouse architecture in production.
• A transformation framework with built-in testing, documentation, and lineage, such as dbt.
• Asset-based or flow-based orchestration (Dagster, Prefect) in addition to AWS-native workflow services.
• Lineage and catalog tooling: AWS Glue Data Catalog, OpenLineage/Marquez, DataHub, OpenMetadata, Amundsen, Collibra, or Alation.
• Streaming and CDC at scale: Kinesis, Kafka/MSK, Debezium, or equivalent, including exactly-once and ordering concerns.
• Distributed query and processing engine tuning: Spark, Trino, or Athena.
• Exposure to serving data to LLMs or AI agents, such as MCP servers, semantic layers, text-to-SQL, or retrieval and vector stores, plus awareness of the failure modes (ambiguous columns, unbounded scans, leaked sensitive fields, stale context).
• Infrastructure as code (Terraform, CloudFormation, or CDK) and containerized workloads.
• Cost awareness and optimization: attributing and reducing storage, scan, and compute cost.
• Familiarity with the downstream BI layer and how modeling decisions constrain it.
• Domain exposure to e-commerce, print or custom manufacturing, or high-volume order data: orders, SKUs and product configuration, pricing, fulfillment, and marketing attribution.