Data Engineer

Whr Global Consulting; Philippines; On-site; PHP 1,339,000 - 2,009,000; Full time

Job summary

Whr Global Consulting in Quezon City seeks an Azure Data Engineer to design, develop, and manage data pipelines and architectures that support business analytics and decision-making.

The role requires 3-5 years of experience, on-site work in Quezon City, and a strong background in data processing frameworks and cloud infrastructure. Proficiency in SQL, DAX, Python, and related tools is preferred.

Qualifications

• Degree in Computer Engineering / Data Science / Statistics / Physics or IT related field.
• Knowledge in Agile methodology (e.g. Scrum & Kanban)
• Experience using cloud platforms (Azure)
• Experience with SQL, DAX, GitHub, Databricks, AWS, PowerBI, Linux

Responsibilities

• Design, build, and maintain scalable data pipelines from various sources.
• Maintain databases, data warehouses, and data lakes ensuring integrity and scalability.
• Implement ETL/ELT processes to automate data collection and transformation.
• Integrate data from APIs, cloud storage, third-party systems, and databases.
• Optimize data architectures for performance, scalability, and cost efficiency.
• Collaborate with data team to deliver clean, accessible data solutions.
• Implement data validation and monitoring to ensure accuracy and integrity.
• Manage cloud infrastructure (AWS, Azure) to support storage, processing, and retrieval.
• Automate repetitive tasks using Python and SQL and workflow tools.
• Document data architectures and pipelines for IT teams; provide troubleshooting support.
• Engage with business and tech partners to define requirements and acceptance criteria.
• Conduct live verification and testing to avoid production errors.

Job description

On-site - Quezon City 3-5 Yrs Exp Bachelor Full-time

Job Description

Job Summary

The Azure Data Engineer will be responsible for designing, developing, and managing data pipelines and architectures. S/he will ensure that data flows seamlessly from multiple sources into databases, data lakes, or warehouses and is processed in a way that supports business analysis and decision-making. The ideal candidate will have strong software engineering skills, experience with data processing frameworks, and the ability to optimize and scale large data systems.

Essential Duties and Responsibilities

• Design, build, and maintain scalable data pipelines that support data collection, transformation, and integration from various sources.
• Maintain databases, data warehouses, data lake ensuring data integrity, scalability, and reliability.
• Implement efficient ETL/ELT (Extract, Transform, Load) processes to automate data collection, cleaning, and transformation tasks.
• Integrate data from various internal and external sources, including APIs, cloud storage, third-party systems, and databases.
• Optimize data architectures for performance, scalability, and cost efficiency. Ensure fast and reliable data retrieval.
• Work closely with the data team to understand data needs and deliver reliable, clean, and accessible data solutions.
• Implement data validation and monitoring processes to ensure data accuracy and integrity. Identify and troubleshoot data issues and bottlenecks.
• Manage and optimize cloud-based infrastructure (e.g., AWS, Azure) to support data storage, processing, and retrieval.
• Automate repetitive tasks and processes using scripting languages (e.g., Python and SQL) and workflows.
• Document data architectures, pipelines, and workflows that will be understood by the Information Technology (IT) department. Provide technical support and troubleshooting to the team.
• Demonstrating a strong track record of interacting with the business and technology partners to drive the requirement, technical process into acceptance criteria.
• Keen on performing user live verification and testing to avoid errors before going to the production environment.
• Provide support for data extraction and reports as required.

Performance Indicators

• Data Pipeline Efficiency and Reliability
• Data Quality and Integrity
• Scalability of Systems
• Cost Efficiency of Data Infrastructure
• Automation and Process Improvement
• Collaboration and Communication
• System Downtime and Availability
• Documentation and Knowledge Sharing
• Innovation and Problem-Solving

Qualifications

• Degree in Computer Engineering / Data Science / Statistics / Physics or any other related field in IT.
• Knowledge in Agile methodology (e.g. Scrum & Kanban)
• Experienced in using cloud computing platforms (Azure)
  • SQL
  • DAX
  • Github; Databricks; AWS; PowerBI; Linux
• Other Qualifications
  • ETL/ELT; Integration; API; Statistics; Code/Performance Optimization; Data Wrangling; Data Pipelines; Data Modeling; Data Quality; Data Governance
• Nice to have but not required

Working Location

Quezon City, National Capital Region, Philippines