Data Engineer

Dempsey Resource Management Inc.; Quezon City; Hybrid; PHP 1,116,000 - 1,674,000; Full time

Job summary

A leading staffing firm is seeking a Cloud Data Engineer to design and maintain cloud-based data pipelines in Quezon City. The ideal candidate has extensive experience in cloud data integration, strong Python skills, and proficiency with big data formats like Parquet and ORC. This hybrid role requires onsite presence and offers a project-based engagement with potential extensions based on performance.

Qualifications

• At least 5 years in cloud data engineering with data integration.
• Strong skills in Python for API integration and data libraries.
• Proficiency with big data storage formats and SQL.
• Experience with data modeling, including NoSQL.

Responsibilities

• Design and maintain cloud-based data pipelines.
• Integrate data from various sources.
• Develop Python code for data processing.
• Optimize data storage using big data formats.
• Implement robust data models.

Skills

Cloud data integration

Python programming

Data modeling

SQL proficiency

Data orchestration

Tools

AWS Services (Glue, IAM, Lambda)

Apache Airflow

Infrastructure as Code (CloudFormation or CDK)

Big data formats (Parquet, ORC)

Job description

Overview

Design, build, and maintain cloud-based data pipelines and workflows that support analytics and operational systems. Integrate data from various sources using APIs and cloud services. Develop clean, efficient, and test-driven code in Python for data ingestion and processing. Optimize data storage and retrieval using big data formats like Apache Parquet and ORC. Implement robust data models, including relational, dimensional, and NoSQL models. Collaborate with cross-functional teams to gather and refine requirements and deliver high-quality solutions. Deploy infrastructure using Infrastructure as Code (IaC) tools such as AWS CloudFormation or CDK. Monitor and orchestrate workflows using Apache Airflow or Dagster. Follow best practices in data governance, quality, and security.

Responsibilities

• Design, build, and maintain cloud-based data pipelines and workflows to support analytics and operational systems.
• Integrate data from various sources using APIs and cloud services.
• Develop clean, efficient Python code for data ingestion and processing with test-driven development.
• Optimize data storage and retrieval using big data formats (Parquet, ORC).
• Implement robust data models, including relational, dimensional, and NoSQL models.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions.
• Deploy infrastructure using IaC tools (AWS CloudFormation or CDK).
• Monitor and orchestrate data workflows using Apache Airflow or Dagster.
• Follow best practices in data governance, quality, and security.

Qualifications & Experience

• Experience: At least 5 years in a cloud data engineering role working on data integration, processing, and transformation using open-source languages (e.g., Python) and cloud technologies.
• Strong programming skills in Python for API integration and data libraries, with emphasis on quality and test-driven development.
• Proficiency with big data storage formats (Apache Parquet, ORC) and knowledge of optimization strategies.
• Proficiency with SQL.
• Experience with data modeling: relational, dimensional, and NoSQL modeling.
• Working knowledge of IaC on AWS (CloudFormation or CDK).
• Working knowledge of AWS Services: Glue, IAM, Lambda, DynamoDB, Step Functions, S3, CloudFormation or CDK. Nice-to-have: Athena, Kinesis, MSK, MWAA, SQS.
• Experience with data orchestration of pipelines: Apache Airflow or Dagster. Nice-to-have: data streaming (Kinesis, Kafka), Apache Spark.
• Client-facing experience, multi-cultural team experience, technical leadership, or team leadership is a plus.

Working Arrangements

• Must be willing to work in a hybrid setup, with onsite reporting to UP Ayala Technohub, Quezon City.
• Available ASAP.
• Engagement is project-based for 6 months, with a possibility of extension for good performance.
• Work schedule is mid-shift and graveyard rotation.
• The role observes U.S. holidays instead of Philippine holidays.