Data Engineer Lead

Viventis Search Asia; Taguig; On-site; PHP 1,800,000 - 3,400,000; Full time

Job summary

Viventis Search Asia is seeking an experienced Lead Data Engineer to transform legacy SSIS/SSRS pipelines into modular, cloud-native data architectures. You will design reusable data patterns, validate new technologies through POCs/POVs, and leverage AI-enabled tools to accelerate engineering work.

You will collaborate across data architects, platform engineers, analysts, and product owners to deliver scalable data pipelines on Azure and GCP while mentoring junior engineers and optimizing cost

Qualifications

• 8+ years of data engineering experience, including at least 3+ years in a lead or senior role.
• Proven ability to design and implement data design patterns (CDC, SCD, Medallion, Data Vault, streaming, batch).
• Deep expertise with Databricks, Apache Spark, dbt, Fivetran, Census, Airflow, and Kafka.
• Solid experience across Azure and/or GCP (e.g., Synapse, Data Factory, BigQuery, Pub/Sub).
• Hands-on experience modernizing legacy ETL (SSIS/SSRS) workloads into cloud-native pipelines.
• Demonstrated ability to build POCs/POVs validating new tools or architectures.
• Working knowledge of AI-assisted engineering tools and programming in SQL and Python/Scala/Java.
• Strong problem-solving, architectural thinking, and collaboration skills.

Responsibilities

• Modernize legacy ETL pipelines using Databricks, dbt, Fivetran, and Airflow.
• Architect reusable data design patterns for ingestion, transformation, and consumption layers across the Lakehouse.
• Develop and lead POCs/POVs to validate new technologies and influence enterprise roadmap.
• Leverage AI-enabled tools to accelerate engineering and automate testing, documentation, and optimization.
• Apply ML for operational intelligence to detect anomalies and optimize compute/scheduling.
• Drive CI/CD, version control, peer reviews, and observability across the data platform.
• Collaborate with data architects, platform engineers, analysts, and product owners to translate needs into solutions.
• Mentor data engineers and promote modern data engineering practices.
• Optimize Spark workloads, storage tiers, and orchestration in Azure and GCP environments.
• Administer and test data system reliability and monitor performance.

Skills

Data engineering

Lead role

SQL

Python

Scala

Java

Databricks

Apache Spark

dbt

Fivetran

Airflow

Kafka

Azure/GCP

Data Vault/Medallion

Education

Bachelor's degree

Tools

Synapse

Data Factory

BigQuery

Pub/Sub

Delta Live Tables

Iceberg

Job description

About the role

Lead the transformation of legacy ETL pipelines into modular, high-performance solutions using Databricks, dbt, Fivetran, and Airflow. You will architect reusable data design patterns, develop and lead POCs/POVs to validate new technologies, and leverage AI-enabled tools to accelerate engineering. This role involves designing and implementing complex data pipelines, modernizing SSIS/SSRS workloads into cloud-native architectures, and optimizing performance and cost across Azure and GCP environments.

Key responsibilities

• Modernize legacy ETL pipelines: Lead the transformation of SSIS/SSRS workloads into modular, high-performance pipelines using Databricks, dbt, Fivetran, and Airflow
• Architect reusable data design patterns: Define and implement standardized frameworks for ingestion, transformation, curation, and consumption layers across the Lakehouse
• Develop and lead POCs/POVs: Experiment with new technologies (e.g., Delta Live Tables, Iceberg, streaming ingestion, AI-driven observability) to validate architecture choices and influence the enterprise roadmap
• Leverage AI to accelerate engineering: Use AI-enabled tools like Databricks Assistant, Cursor AI, GitHub Copilot, and dbt Mesh AI tests for code generation, automated testing, documentation, and pipeline optimization
• Apply ML for operational intelligence: Integrate predictive models to detect pipeline anomalies, data drift, and optimize compute and scheduling
• Enforce engineering excellence: Drive CI/CD, version control, peer reviews, and observability practices across the data platform
• Collaborate cross-functionally: Partner with data architects, platform engineers, analysts, and business product owners to translate business needs into technical solutions
• Mentor data engineers: Provide technical guidance, foster continuous learning, and help the team adopt modern data engineering best practices
• Optimize performance and cost: Continuously tune Spark workloads, storage tiers, and orchestration logic across Azure and GCP environments
• Administer and test the reliability of data systems; monitor performance and stability as requirements change

About you

• Bachelor's degree
• 8+ years of experience in data engineering or related technical fields, with at least 3+ years in a lead or senior role
• Proven experience designing and implementing data design patterns (e.g., CDC, SCD, Medallion, Data Vault, streaming, and batch patterns)
• Deep expertise with Databricks, Apache Spark, dbt, Fivetran, Census, Airflow, and Kafka
• Solid experience across Azure and/or GCP (e.g., Synapse, Data Factory, BigQuery, Pub/Sub)
• Hands-on experience modernizing legacy ETL (SSIS/SSRS) workloads into cloud-native pipelines
• Demonstrated ability to build POCs and POVs that validate new tools, frameworks, or architectures
• Working knowledge of AI-assisted engineering tools for development, observability, or optimization
• Proficiency in SQL and one programming language (Python, Scala, or Java)
• Strong problem-solving, architectural thinking, and collaboration skills