Data Engineer Consultant

AIA Shared Services; Makati; On-site; PHP 900,000 - 1,500,000; Full time

Job summary

MediCard Phils., Inc. is seeking a Data Platform Engineer to design and maintain scalable data platforms on Azure, develop ETL/ELT pipelines with Databricks, and ensure data quality and governance across all data flows.

You will collaborate with data analysts, data scientists, and BI teams to deliver analytics-ready data and support modernization initiatives in a healthcare-focused environment.

Qualifications

• Experience designing data platforms on Azure
• Proficient in building and maintaining ETL/ELT pipelines
• Strong knowledge of data modeling for analytics
• Experience with Delta Lake and Spark technologies
• Familiarity with data quality, validation, and governance practices

Responsibilities

• Design and maintain scalable data platforms on Microsoft Azure
• Develop and orchestrate ETL/ELT pipelines using Azure Data Factory and Databricks
• Build data transformations using Databricks (PySpark, Spark SQL) and Delta Lake
• Model and store data in Azure Data Lake for analytics and reporting
• Ensure data quality, governance, and security across data pipelines
• Collaborate with analysts, data scientists, and BI developers
• Document architectures, pipelines, and operational procedures
• Implement CI/CD and automation for data engineering workflows
• Monitor and optimize data pipelines for reliability and cost efficiency
• Stay updated with Azure and Databricks best practices

Skills

Azure Databricks

ETL/ELT pipelines

Data modeling

Delta Lake

PySpark

Spark SQL

Azure Data Factory

Data quality

Data governance

Cloud architecture

Tools

Databricks

Azure Data Factory

Azure Data Lake

API integration

Job description

MediCard Phils., Inc. is one of the country's leading HMO and the only HMO founded and run by Doctors. Since its inception, the concept of service-oriented total health care has been the molding ideal of MediCard. The competition is vast, and the benefits being offered by the competitors are tempting. However, MEDICard has taken the lead in providing innovative and productive ideas that cut down the cost of health maintenance without compromising its quality. MediCard now boasts of more than half a million members and over 54,000 accredited doctors in over 1,000 hospitals and clinics nationwide. It also operates 16 MediCard free-standing clinics that provide services at par with those offered by hospitals minus the confinement. MediCard is currently looking for assertive, dynamic and energetic individuals to fill up the following vacancy:

• Responsible for capturing data from various sources into the system for data analytics
• Apply data extraction, transformation and loading techniques in order to connect large data sets from a variety of sources
• Create data collection frameworks for structured and unstructured data
• Develop and maintain infrastructure systems such as data warehouses, data lakes and data access application programming interfaces (APIs)
• Define data standards, develop data catalogues and ensure data quality (map Data Engineer, Data Management, Data Architect to this job profile)

1. Data Platform Design aligned with Modern Data Architecture:

• Design, build, and maintain scalable, secure, and high‑performance data platforms on Microsoft Azure.
• Build end‑to‑end data solutions leveraging Azure Databricks, Azure Data Factory, Azure Data Lake, and related Azure services.
• Ensure data designs follows best practices for reliability, cost optimization, performance, and extensibility.

2. ETL / ELT Pipeline Development:

• Develop, orchestrate, and maintain robust ETL/ELT pipelines using Azure Data Factory and Databricks.
• Implement batch and incremental data ingestion from diverse sources including databases, APIs, SaaS platforms, and data streams.
• Optimize pipelines for reliability, scalability, and performance while minimizing data latency.

3. Databricks Engineering & Optimization:

• Develop data transformation, enrichment, and aggregation logic using Databricks (PySpark, Spark SQL, Delta Lake).
• Implement Delta Lake best practices including schema enforcement, versioning, time travel, and performance tuning.
• Optimize Databricks clusters, jobs, and notebooks for cost efficiency and processing performance.

4. Data Modeling & Storage:

• Design and implement data models optimized for analytics, reporting, and downstream consumption.
• Manage structured, semi‑structured, and unstructured data in Azure Data Lake and cloud‑based storage systems.
• Apply partitioning, indexing, and compression strategies to improve query performance and storage efficiency.

5. Data Quality & Reliability:

• Implement data validation, quality checks, and reconciliation processes within ETL pipelines.
• Monitor data freshness, completeness, accuracy, and consistency across data flows.
• Troubleshoot and resolve data pipeline failures, data integrity issues, and performance bottlenecks.

6. Cloud Security, Governance & Compliance:

• Apply Azure security best practices including identity management, RBAC, network security, and encryption.
• Ensure data pipelines and platforms comply with organizational governance standards and regulatory requirements.
• Collaborate with security and platform teams to manage access control, secrets, and audit logging.

7. Automation, Monitoring & DevOps:

• Implement CI/CD pipelines for data engineering code and infrastructure using Azure DevOps or similar tools.
• Automate deployments, testing, and configuration management for data platforms and pipelines.
• Set up monitoring, alerting, and logging for Databricks jobs and ADF pipelines to ensure operational stability.

8. Collaboration & Stakeholder Engagement:

• Work closely with data analysts, data scientists, BI developers, and business stakeholders to understand data requirements.
• Translate business needs into technical data solutions that support analytics and decision‑making.
• Participate in design reviews, sprint planning, and cross‑functional working sessions.

9. Documentation & Knowledge Management:

• Document data architectures, pipeline designs, transformation logic, and operational procedures.
• Maintain technical documentation for Databricks notebooks, ADF pipelines, and data models.
• Contribute to shared repositories, best‑practice guides, and team knowledge bases.

10. Continuous Improvement & Innovation:

• Stay current with Azure, Databricks, and data engineering best practices and emerging technologies.
• Proactively identify opportunities to improve performance, scalability, and cost efficiency.
• Support modernization initiatives, cloud adoption, and advanced analytics use cases.

"Better starts with people who reimagine what is possible and turn ideas into meaningful action. At AIA, you will contribute in your own way, help shape outcomes that matter and collaborate with colleagues across disciplines, teams, markets and levels. We will support you to broaden your perspective, strengthen your capabilities and grow your impact. Together, we challenge assumptions, learn from one another and find better ways forward. Bring your ambition, experience and ideas to AIA, and help shape what "better" looks like. View our AIA LinkedIn page Bring your difference to AIA