Data Engineer: ETL Pipelines, Lakehouse & Kubernetes

ADECCO PERSONNEL PTE LTD; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

ADECCO PERSONNEL PTE LTD is seeking a Data Engineer to design and build scalable ETL/ELT data pipelines using Python, PySpark and Spark. You will manage lakehouse architectures with Iceberg tables and implement metadata and partitioning strategies.

Responsibilities include deploying pipelines with Kubernetes, Docker, CI/CD, and DevOps practices in on-prem environments, providing L3 support, and ensuring data security and governance while collaborating with stakeholders.

Qualifications

• Demonstrated experience in data engineering/pipeline development with Python and SQL Server.

Responsibilities

• Design and develop scalable ETL/ELT data pipelines using Python, PySpark, MSBI and Apache Spark.
• Build and manage lake house/data platforms, including data modelling, partitioning, metadata and Iceberg tables.
• Deploy and manage data pipelines using Kubernetes, Docker, CI/CD and DevOps practices in on-premise environments.
• Provide L3 support, optimise data platform performance, ensure security/governance and collaborate with technical and business stakeholders.

Skills

Python

SQL

MSBI

Tools

Kubernetes

Docker

Apache Spark/PySpark

Apache Airflow

Iceberg

MongoDB

Cassandra

Git

IaC

Job description

ADECCO PERSONNEL PTE LTD is seeking a Data Engineer to design and build scalable ETL/ELT data pipelines using Python, PySpark and Spark. You will manage lakehouse architectures with Iceberg tables and implement metadata and partitioning strategies.

Responsibilities include deploying pipelines with Kubernetes, Docker, CI/CD, and DevOps practices in on-prem environments, providing L3 support, and ensuring data security and governance while collaborating with stakeholders.