Data Engineer Consultant

AIA (Web); Makati; On-site; PHP 600,000 - 900,000; Full time

Job summary

MediCard Phils., Inc. is seeking a data platform engineer to design, build and maintain Azure-based data platforms. The role involves end-to-end data solutions using Azure Databricks, ADF and Data Lake, with a focus on scalable, secure pipelines and data modeling.

You will collaborate with data analysts, BI developers and business stakeholders to translate requirements into robust data architectures and pipelines, ensuring data quality, governance and optimized performance.

Qualifications

• Capture data from various sources into the system for data analytics.
• Apply data extraction, transformation and loading techniques to connect large data sets.
• Create data collection frameworks for structured and unstructured data.
• Develop and maintain infrastructure systems such as data warehouses, data lakes and APIs.
• Define data standards, develop data catalogues and ensure data quality.

Responsibilities

• Design, build, and maintain scalable Azure-based data platforms.
• Build end-to-end data solutions using Azure Databricks, ADF, and Data Lake.
• Develop and orchestrate ETL/ELT pipelines across diverse sources.
• Model data for analytics and downstream consumption.
• Implement data validation and monitoring within pipelines.
• Apply security, governance and compliance best practices.
• Collaborate with analysts, data scientists and BI developers.
• Document architectures, pipelines and data models.
• Stay current with Azure and data engineering best practices.
• Support modernization and cloud adoption initiatives.

Skills

ETL/ELT pipelines

Azure Databricks

Azure Data Factory

Delta Lake

PySpark

Spark SQL

Data Modeling

Azure Data Lake

DevOps CI/CD

Data quality

Tools

Azure Databricks

Azure Data Factory

Delta Lake

Azure DevOps

Databricks

Job description

MediCard Phils., Inc.is one of the country's leading HMOand the only HMOfounded and run by Doctors. Since its inception, the concept of service-oriented total health care has been the molding ideal of MediCard. The competition is vast, and the benefits being offered by the competitors are tempting. However, MEDICard has taken the lead in providing innovative and productive ideas that cut down the cost of health maintenance without compromising its quality.

MediCardnow boasts of more than half a million members and over 54,000 accredited doctors in over 1,000 hospitals and clinics nationwide. It also operates 16 MediCard free-standing clinics that provide services at par with those offered by hospitals minus the confinement.

MediCardis currentlylooking for assertive, dynamic and energetic individuals to fill up the following vacancy:

• Responsible for capturing data from various sources into the system for data analyticsApply data extraction, transformation and loading techniques in order to connect large data sets from a variety of sources
• Create data collection frameworks for structured and unstructured data
• Develop and maintain infrastructure systems such as data warehouses, data lakes and data access application programming interfaces (APIs)
• Define data standards, develop data catalogues and ensure data quality
• (map Data Engineer, Data Management, Data Architect to this job profile)

1. Data Platform Design aligned with Modern Data Architecture:

• Design, build, and maintain scalable, secure, and high‑performance data platforms on Microsoft Azure.
• Build end‑to‑end data solutions leveraging Azure Databricks, Azure Data Factory, Azure Data Lake, and related Azure services.
• Ensure data designs follows best practices for reliability, cost optimization, performance, and extensibility.

2. ETL / ELT Pipeline Development:

• Develop, orchestrate, and maintain robust ETL/ELT pipelines using Azure Data Factory and Databricks.
• Implement batch and incremental data ingestion from diverse sources including databases, APIs, SaaS platforms, and data streams.
• Optimize pipelines for reliability, scalability, and performance while minimizing data latency.

3. Databricks Engineering & Optimization:

• Develop data transformation, enrichment, and aggregation logic using Databricks (PySpark, Spark SQL, Delta Lake).
• Implement Delta Lake best practices including schema enforcement, versioning, time travel, and performance tuning.
• Optimize Databricks clusters, jobs, and notebooks for cost efficiency and processing performance.

4. Data Modeling & Storage:

• Design and implement data models optimized for analytics, reporting, and downstream consumption.
• Manage structured, semi‑structured, and unstructured data in Azure Data Lake and cloud‑based storage systems.
• Apply partitioning, indexing, and compression strategies to improve query performance and storage efficiency.

5. Data Quality & Reliability:

• Implement data validation, quality checks, and reconciliation processes within ETL pipelines.
• Monitor data freshness, completeness, accuracy, and consistency across data flows.
• Troubleshoot and resolve data pipeline failures, data integrity issues, and performance bottlenecks.

6. Cloud Security, Governance & Compliance:

• Apply Azure security best practices including identity management, RBAC, network security, and encryption.
• Ensure data pipelines and platforms comply with organizational governance standards and regulatory requirements.
• Collaborate with security and platform teams to manage access control, secrets, and audit logging.

7. Automation, Monitoring & DevOps:

• Implement CI/CD pipelines for data engineering code and infrastructure using Azure DevOps or similar tools.
• Automate deployments, testing, and configuration management for data platforms and pipelines.
• Set up monitoring, alerting, and logging for Databricks jobs and ADF pipelines to ensure operational stability.

8. Collaboration & Stakeholder Engagement:

• Work closely with data analysts, data scientists, BI developers, and business stakeholders to understand data requirements.
• Translate business needs into technical data solutions that support analytics and decision‑making.
• Participate in design reviews, sprint planning, and cross‑functional working sessions.

9. Documentation & Knowledge Management:

• Document data architectures, pipeline designs, transformation logic, and operational procedures.
• Maintain technical documentation for Databricks notebooks, ADF pipelines, and data models.
• Contribute to shared repositories, best‑practice guides, and team knowledge bases.

10. Continuous Improvement & Innovation:

• Stay current with Azure, Databricks, and data engineering best practices and emerging technologies.
• Proactively identify opportunities to improve performance, scalability, and cost efficiency.
• Support modernization initiatives, cloud adoption, and advanced analytics use cases.