Lead Data Engineer

Yondu, Inc.; Taguig; Presencial; PHP 1 000 000 - 1 800 000; Tempo integral

Resumo da oferta

Yondu, Inc. in Manila, Philippines invites an experienced Lead Data Engineer to design, build, and optimize scalable data infrastructure for analytics needs.

You will lead a team of data engineers and partner with data scientists, analysts, and business stakeholders to deliver reliable, governance-backed data solutions. The role focuses on ETL/ELT pipelines, data lake and warehouse architectures, and modern tools like Spark, Kafka, Airflow, and DBT, with cloud platforms (AWS/Azure/GCP).

Qualificações

• Bachelor's or Master's degree in IT, CS, or engineering.
• 7+ years in data engineering with 2-3 years in a leadership or senior technical role.
• Proven expertise in enterprise data pipelines and architectures.
• Strong cloud experience with AWS, Azure, or GCP.
• Proficient in SQL and programming (Python, Java, Scala).
• Hands-on with Spark, Hadoop, and Kafka.
• Knowledge of data warehousing (Snowflake, Redshift, BigQuery, Synapse).
• Experience with workflow orchestration (Airflow, Prefect, Luigi).
• Solid understanding of DevOps/CI/CD for data pipelines.

Responsabilidades

• Lead and mentor a team of data engineers.
• Define and enforce data engineering standards and guidelines.
• Collaborate with data scientists, analysts, and business stakeholders.
• Design and maintain high-performance ETL/ELT pipelines and workflows.
• Build and optimize data lake, data warehouse, and streaming data solutions.
• Ensure support for structured, semi-structured, and unstructured data.
• Develop data transformation and preparation processes for analytics-ready datasets.
• Implement validation, anomaly detection, and automated quality checks.

Conhecimentos

SQL

Python/Java/Scala

Spark

Hadoop

Kafka

Airflow

SQL/NoSQL

Cloud platforms

Data governance

Leadership

Mentoring

Communication

Formação académica

Bachelor's or Master's in IT/CS/Engineering

Ferramentas

Snowflake

Redshift

BigQuery

Synapse

Airflow

Prefect

Luigi

Spark

Hadoop

Kafka

AWS

Azure

GCP

Descrição da oferta de emprego

The Lead Data Engineer is responsible for designing, developing, and optimizing data infrastructure to ensure reliable pipelines, scalable architecture, and high-quality data for analytics and business needs. The role includes leading a team of data engineers, collaborating with cross-functional stakeholders, and implementing best practices in data transformation, validation, governance, and cloud-based solutions.

DUTIES AND RESPONSIBILITIES:

• Lead and mentor a team of data engineers, fostering collaboration, innovation, and continuous improvement.
• Define and enforce data engineering standards, coding practices, and architectural guidelines.
• Partner with data scientists, analysts, and business stakeholders to translate requirements into scalable solutions.
• Design and maintain high-performance ETL/ELT pipelines and workflows.
• Build and optimize data lake, data warehouse, and streaming data solutions.
• Ensure systems support structured, semi-structured, and unstructured data sources.
• Develop data transformation and preparation processes to deliver clean, reliable, analytics-ready datasets.
• Implement validation, anomaly detection, and automated quality checks to ensure accuracy and consistency.
• Define and execute data remediation strategies to resolve issues with minimal business disruption.
• Implement and manage cloud-based data platforms (AWS, Azure, or GCP).
• Apply modern frameworks (e.g., Spark, Kafka, Airflow, DBT) for efficient data processing.
• Oversee data modeling, schema design, and database optimization (SQL/NoSQL).
• Ensure accuracy, consistency, and availability of data across platforms.
• Implement governance, lineage, validation frameworks, and security best practices.
• Monitor, troubleshoot, and optimize pipelines to meet SLAs and performance benchmarks.
• Stay current with emerging data engineering and analytics technologies.
• Recommend improvements to systems, tools, and processes to enhance scalability, reliability, and quality.
• Other job-related activities that may be assigned from time to time.

JOB SPECIFICATIONS:

• Education - At least graduate with a Bachelor's or Master's Degree in IT, Computer Science, Engineering, or any related course.
• Related Work Experience -
• 7+ years of experience in data engineering, with at least 2-3 years in a leadership or senior technical role.
• Proven expertise in building and managing enterprise-level data pipelines and architectures.
• Strong experience with one or more cloud platforms (AWS, Azure, GCP).
• Knowledge - Knowledgeable in the following:
• Proficiency in SQL and at least one programming language (Python, Java, Scala).
• Hands-on experience with big data tools (Spark, Hadoop, Kafka).
• Strong knowledge of data transformation, preparation, validation, and remediation techniques.
• Familiarity with data warehousing solutions (Snowflake, Redshift, BigQuery, Synapse).
• Experience with workflow orchestration (Airflow, Prefect, Luigi).
• Solid understanding of DevOps and CI/CD for data pipelines.
• Skills
• Excellent communication and interpersonal skills to collaborate effectively with developers, analysts, and business stakeholders.
• Strong ability to document data requirements and translate them into technical solutions.
• Proven leadership and mentoring capabilities to guide and develop team members.
• Ability to work seamlessly across technical and non-technical teams.
• Strong problem-solving mindset with focus on detail, data quality, and scalability.
• Ability to thrive in a fast-paced environment and meet tight SLAs.