Data Engineer

Pro Integrate PH; Taguig; Hybrid; PHP 1,200,000 - 1,600,000; Full time

Benefits offered by this job

HMO

Job Training

Professional Development

Job summary

A leading data solutions company in Taguig is seeking an experienced Data Engineer to design and manage data products and pipelines using Azure technologies. Candidates should possess over 7 years of experience, with a strong foundation in SQL and data engineering principles. The role involves collaboration with various teams to ensure data quality and optimize performance while implementing CI/CD processes. This hybrid position offers opportunities for professional development and health benefits.

Qualifications

• 7+ years of experience as a Data Engineer or similar role.
• Proficiency in SQL DML for querying databases.
• Strong problem-solving skills and attention to detail.

Responsibilities

• Design and maintain data pipelines utilizing Azure technologies.
• Collaborate with stakeholders to understand data requirements.
• Monitor and troubleshoot data pipeline issues.

Skills

Data engineering

SQL

Azure

Python

Spark

Data validation

CI/CD

Problem-solving

Education

Bachelor’s degree in computer science or related field

Tools

Azure Data Lake Storage

Azure DevOps

SQL Server

PostgreSQL

PySpark

Job description

Hybrid - Taguig 5-10 Yrs Exp Bachelor Full-time

Job Description

This position designs data products and data pipelines that are resilient to change, modular, flexible, scalable, reusable, and cost effective. Utilizes software engineering principles to deploy and maintain fully automated data transformation pipelines that combine a large variety of storage and computation technologies to handle a distribution of data types and volumes in support of data architecture design.

Government Mandated Benefits

Insurance Health & Wellness

HMO

Professional Development

Job Training, Professional Development

Responsibilities include, but are not limited to, the following:

• Utilize Azure data storage accounts for organizing and maintaining data pipeline outputs (e.g., Azure Data Lake Storage Gen 2 & Azure Blob storage).
• Collaborate with data scientists, data analysts, data architects and other stakeholders to understand data requirements and deliver high-quality data solutions.
• Optimize data pipelines in the Azure environment for performance, scalability, and reliability.
• Ensure data quality and integrity through data validation techniques and frameworks.
• Develop and maintain documentation for data processes, configurations, and best practices.
• Monitor and troubleshoot data pipeline issues to ensure timely resolution.
• Stay current with industry trends and emerging technologies to ensure our data solutions remain cutting‑edge.
• Manage the CI/CD process for deploying and maintaining data solutions.

Required Qualifications:

• Bachelor’s degree in computer science, engineering, or a related field (or equivalent experience) and ability to demonstrate high proficiency in programming fundamentals.
• Proven experience (7+ years) as a Data Engineer or similar role dealing with data and ETL processes.
• Experience utilizing SQL DML to query modern RDBMS efficiently (e.g., SQL Server, PostgreSQL).
• Strong understanding of software engineering principles and how they apply to data engineering (e.g., CI/CD, version control, testing).
• Experience with big data technologies (e.g., Spark).
• Strong problem‑solving skills and attention to detail.
• Excellent communication and collaboration skills.

Preferred Qualifications:

• Learning agility
• Technical leadership
• Consulting and managing business needs
• Strong experience in Python (preferred but other languages such as Scala, Java, C#, etc. are accepted).
• Experience building spark applications utilizing PySpark.
• Experience with file formats such as Parquet, Delta, Avro.
• Experience efficiently querying API endpoints as a data source.
• Understanding of the Azure environment and related services such as subscriptions, resource groups, etc.
• Understanding of Git workflows in software development.
• Using Azure DevOps pipeline and repositories to deploy and maintain solutions.
• Understanding of Ansible and how to use it in Azure DevOps pipelines.