Data Engineer: GenAI-Powered Data Pipelines

re-zoo-me; Singapore; On-site; SGD 120,000 - 170,000; Full time

Job summary

re-zoo-me is seeking a seasoned Data Engineer in Singapore to design, develop and maintain scalable ETL/ELT data pipelines for large datasets. You will implement batch and real-time processing using Python, PySpark, Spark SQL and Scala, and work with Kafka, Kinesis, Spark Streaming, AWS Glue and Airflow.

Design data lakes and warehouses, integrate with REST APIs and cloud databases, and collaborate with data scientists and software engineers to deliver enterprise-grade data solutions using

Qualifications

• Strong professional experience in Big Data and cloud computing.
• Experience with large-scale enterprise data platforms and pipelines.
• Strong programming and SQL skills.
• Experience with cloud-based data engineering and modern data processing frameworks.
• Experience with AI/ML, NLP or GenAI is advantageous.

Responsibilities

• Design, develop and maintain scalable ETL/ELT data pipelines.
• Develop high-performance data processing solutions using Python, PySpark, Spark SQL and Scala.
• Build batch and real-time pipelines using Kafka, Kinesis, Spark Streaming, AWS Glue and Airflow.
• Design and implement data lakes, data warehouses and cloud-based processing platforms.
• Work with Databricks, Hadoop, Hive, Cloudera, Snowflake for large-scale processing and analytics.
• Integrate data platforms with REST APIs, GraphQL and enterprise apps.
• Implement CI/CD and DevOps practices using Jenkins, Git, Docker, Kubernetes and OpenShift.

Skills

Python

SQL

Scala

Java

Shell Scripting

Git

Education

Bachelor's or master's in Computer Science or related

Tools

Databricks

Hadoop

Hive

Cloudera

Snowflake

Kafka

Kubernetes

OpenShift

Jenkins

Airflow

Spark

Spark SQL

Job description

re-zoo-me is seeking a seasoned Data Engineer in Singapore to design, develop and maintain scalable ETL/ELT data pipelines for large datasets. You will implement batch and real-time processing using Python, PySpark, Spark SQL and Scala, and work with Kafka, Kinesis, Spark Streaming, AWS Glue and Airflow.

Design data lakes and warehouses, integrate with REST APIs and cloud databases, and collaborate with data scientists and software engineers to deliver enterprise-grade data solutions using