Azure Data Engineer

JG Summit Holdings Inc.; Pasig; Presencial; PHP 900.000 - 1.500.000; Jornada completa

Descripción de la vacante

JG Summit Holdings Inc. seeks an experienced Data Engineer to architect and maintain scalable data pipelines on Azure, using Databricks and Data Factory. You will design Bronze-Silver-Gold medallion data lakes, model data, and ensure secure, efficient analytics for enterprise reporting.

You will collaborate with cross-functional teams to translate business needs into robust data architectures, monitor pipelines in production, and mentor junior staff in best practices and documentation.

Formación

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field (or equivalent practical experience).
• 5+ years as a Data Engineer with focus on Microsoft Azure ecosystem.
• Deep competency in PySpark and Spark SQL for data processing and transformation.
• Experience developing and orchestrating scalable data pipelines using Azure Databricks and Azure Data Factory (ADF).
• Proven track record designing Medallion Architecture (Bronze, Silver, Gold) within Azure Data Lake for enterprise reporting.
• Strong data modeling and optimization skills, including Azure SQL Database.

Responsabilidades

• Architect and implement scalable data pipelines using Azure Databricks and ADF.
• Monitor, troubleshoot, and optimize ETL/ELT pipelines in production to ensure reliability.
• Collaborate with business stakeholders and analysts to translate requirements into robust data solutions.
• Secure data via RBAC and encryption; document processes and mentor team members.

Conocimientos

PySpark

Spark SQL

Azure Databricks

Azure Data Factory

Medallion Architecture

Azure SQL Database

Data Modeling

RBAC & Data Security

ETL/ELT Pipelines

Cross-functional Collaboration

Educación

Bachelor’s degree in Computer Science or related field

Herramientas

Azure Databricks

Azure Data Factory

Azure SQL Database

Descripción del empleo

• Build Scalable Pipelines: Develop and implement robust, highly scalable data pipelines using Azure Databricks, Azure Data Factory, and other modern Azure tools.
• Design Medallion Architecture: Design and optimize enterprise data storage solutions by implementing a Medallion Architecture (Bronze, Silver, Gold) within Azure Data Lake to efficiently support complex enterprise reporting.
• Execute Advanced Data Transformations: Perform data modeling and create highly efficient data architectures. Competency in PySpark and Spark SQL is a must-have requirement.
• Collaborate Across Teams: Proven experience collaborating with cross-functional teams to translate business data requirements into seamless, integrated technical solutions.
• Production Support & Monitoring: Demonstrate the strong ability to support and monitor ETL/ELT processes in a live production environment, proactively troubleshooting pipelines to ensure accuracy, reliability, and minimal downtime.
• Ensure Data Security: Implement stringent data security measures and access controls to protect sensitive enterprise information.
• Mentor & Document: Document technical processes, establish standard operating procedures, and provide knowledge-sharing and training sessions to the team.
• Drive Innovation: Stay continuously updated with the latest Azure technologies, Databricks advancements, and best practices in modern data engineering.

WHAT IS THE JOB LIKE?

This role acts as the core architect and protector of the company’s data infrastructure. The day-to-day work breaks down into four main areas:

1. The Builder (Focus: ~40%)

The engineer uses Azure Data Factory to extract messy data from various sources and leverages the computing power of Azure Databricks (using PySpark) to clean and transform it. They organize this data into structured, highly efficient architectures—often using a Bronze, Silver, and Gold Medallion setup—within Azure Data Lake or Azure SQL Database. Building this efficiently ensures analysts can run fast enterprise reports while keeping the company's computing costs low.

2. The Firefighter (Focus: ~30%)

Data pipelines occasionally break due to source changes, server timeouts, or corrupted files. The engineer monitors these automated systems, investigates errors, and quickly fixes broken pipelines so business dashboards stay updated.

3. The Translator (Focus: ~15%)

The engineer collaborates regularly with business stakeholders and analysts. They listen to business goals (like tracking real-time sales) and design the technical steps required to find, move, and deliver that specific data seamlessly.

4. The Guardian & Teacher (Focus: ~15%)

The engineer protects the company's data by setting up strict access controls and encrypting sensitive information. As a teacher, they also document technical processes, provide knowledge transfer, and train team members on data systems and best practices.

WHO ARE YOU?

• Experience & Education: Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field (or equivalent practical experience).
• 5+ years of proven experience working as a Data Engineer, with a strong focus on the Microsoft Azure ecosystem.
• Core Technical & Architectural SkillsMust-Have: Deep, hands-on competency in PySpark and Spark SQL for complex data processing and transformation.
• Extensive experience developing and orchestrating scalable data pipelines using Azure Databricks and Azure Data Factory (ADF).
• Proven track record of designing and implementing a Medallion Architecture (Bronze, Silver, Gold) within Azure Data Lake to support downstream enterprise reporting and analytics.
• Solid proficiency in data modeling and optimizing database solutions, including Azure SQL Database.

Operational Expertise

• Demonstrated ability to actively monitor, support, and troubleshoot ETL/ELT pipelines in a live production environment.
• Experience identifying performance bottlenecks, analyzing error logs, and ensuring pipeline reliability and accuracy with minimal downtime.
• Strong understanding of data security best practices, including implementing role-based access control (RBAC) and protecting sensitive enterprise data.
• Worked on SAP systems preferred but not required
• Proven experience collaborating with cross-functional teams (business stakeholders, analysts, and software engineers) to translate business requirements into robust technical solutions.
• Excellent written and verbal communication skills, with the ability to document complex technical processes clearly.
• A collaborative mindset with a willingness to lead knowledge-sharing sessions and mentor team members.