Senior Data Engineer

Permhunt; Manila; On-site; PHP 1,674,000 - 2,567,000; Full time

Job summary

Permhunt, a global leader in digital shopper marketing, seeks a senior data engineer to design and govern data pipelines across OLTP/OLAP. You will shape schemas for Azure SQL, Synapse, and Delta Lake, ensuring security and scalability while supporting analytics and AI initiatives.

You'll own end-to-end data delivery, collaborate with Data Science and ML teams, implement CI/CD for data workloads, and drive data quality and lineage efforts in a multi-tenant SaaS environment.

Qualifications

• 5+ years in data engineering or data architecture.
• Strong Python and SQL with PySpark and pandas.
• Hands-on Azure data services like ADLS Gen2, Synapse, Data Factory.
• Experience with multi-tenant SaaS data architectures and data governance.

Responsibilities

• Design and evolve schemas for OLTP and OLAP workloads on Azure SQL/Synapse/Delta Lake.
• Define data contracts, versioning, and schema evolution; implement CDC and SCD patterns.
• Optimize performance via query tuning, caching, cost controls.
• Build ELT/ETL workflows across batch and streaming using Azure Data Factory/Synapse/Databricks, Event Hubs/Service Bus, Functions, and container workloads.

Skills

Advanced Python

SQL

PySpark

Data modeling

Cloud data services

Tools

Azure Data Factory

Azure Synapse

Databricks

ADLS Gen2

dbt

Job description

Our client is a global leader in digital shopper marketing. Backed by decades of shopper research and deep category expertise, the business uses a proprietary framework and a digital-first approach to help clients navigate an evolving retail landscape.

Job Responsibilities

SQL Databases

• Design and evolve schemas for OLTP and OLAP workloads (Azure SQL, Synapse, Delta Lake), including partitioning, indexing, and row-level security in multi-tenant environments.
• Define and manage data contracts, versioning, and schema evolution; implement CDC and SCD patterns.
• Optimize performance through query tuning, resource configuration, caching strategies, and cost controls.
• Architect and build ELT/ETL workflows across batch and streaming using Azure Data Factory/Synapse/Databricks, Event Hubs/Service Bus, Functions, and containerized workloads (Container Apps/AKS)
• Deliver reliable and observable pipelines (idempotent, retryable, lineage-aware) with defined SLAs/SLOs and operational runbooks.
• Implement CI/CD for data workloads (SQL/dbt projects, PySpark jobs, automated testing) using GitHub Actions and infrastructure-as-code (Terraform/Bicep).
• Design and manage enrichment layers such as standardized identifiers, metadata extraction, taxonomies, embeddings, and third-party data integrations.
• Curate gold/semantic data models for analytics and internal/external data services, owning feature and metric definitions with clear documentation.
• Collaborate with Data Science and ML teams to productionize feature stores, model outputs, drift monitoring, and evaluation datasets.
• Own reference data architecture across ADLS Gen2, Synapse/Databricks, Azure SQL/SQL Server, Cosmos DB, Azure AI Search, Key Vault, and Purview.
• Embed security, privacy, and compliance by default, including encryption, secret management, RBAC/ABAC, retention policies, and GDPR/SOC-aligned controls.
• Drive observability using OpenTelemetry and Azure Monitor/Application Insights, along with data quality checks, freshness SLAs, and lineage tracking.

Examples of What Youll Build

• A scalable media ingestion and enrichment pipeline that validates assets, extracts metadata, generates embeddings, tracks lineage, and publishes analytics-ready and search-ready views.
• A hybrid retrieval layer (vector search plus structured filters) using Cosmos DB and Azure AI Search to support similarity search and recommendation use cases.

Qualifications

• Advanced Python and SQL skills, with experience analyzing complex query plans and working across PySpark and pandas.
• 5+years of experience in data engineering or data architecture with end-to-end ownership of production databases and pipelines.
• Strong hands-on experience with Azure data services, including ADLS Gen2, Data Factory/Synapse/Databricks, Azure SQL/SQL Server, Functions, Event Hubs/Service Bus, and Key Vault.
• Solid foundation in data modeling (star/snowflake, Data Vault, Lakehouse), CDC/SCD patterns, and semantic modeling (dbt or equivalent).
• Proven experience implementing data quality, lineage, and performance/cost guardrails at scale.
• Strong understanding of multi-tenant SaaS architectures, data security, and privacy principles (including core GDPR concepts).

Nice to Have

• Experience with Cosmos DB (including vector capabilities) and Azure AI Search; exposure to image and text embedding pipelines.
• Background in feature stores, MLflow or similar model registries, and real-time inference pipelines.
• Knowledge of SQL Server internals, PolyBase, or serverless SQL; familiarity with PostgreSQL.
• Experience implementing data governance frameworks, Purview, or data product operating models.

Our Clients Technology Stack

• Cloud & Data: Azure (ADLS Gen2, Data Factory, Synapse, Databricks, Functions, Event Hubs, Key Vault, Monitor)
• Languages & Tools: Python (pandas, PySpark, FastAPI for data services), dbt (or equivalent), GitHub Actions, Terraform/Bicep