AdTech Data Engineer: Spark, Flink & Data Pipelines

OPERA TECHNOLOGY SINGAPORE PTE. LTD.; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A leading technology company in Singapore is seeking a skilled Data Engineer to design, develop, and maintain data pipelines. The ideal candidate will have strong proficiency in Python, SQL, and Bash, along with hands-on experience in frameworks like Spark and Flink. Responsibilities include building large-scale data processing workflows and collaborating with cross-functional teams to meet business needs. A Bachelor's degree in Computer Science or a related field is required, along with experience in the AdTech industry.

Qualifications

• Proficiency in Python, SQL, and Bash.
• Experience with Hadoop ecosystem (HDFS, Spark on YARN, Hive).
• Hands-on experience with distributed processing frameworks such as Spark or Flink.
• Familiarity with Kafka, Flume, or similar data ingestion tools.
• Strong analytical skills and attention to data quality.
• Good communication skills and ability to work with technical and non-technical stakeholders.
• Ability to work independently and collaboratively in a multicultural environment.

Responsibilities

• Design, develop, and maintain batch and streaming data pipelines.
• Build and optimise large-scale data processing workflows using Spark, Flink, Hive, and HDFS.
• Work with cross-functional stakeholders to deliver data solutions aligned with business needs.
• Monitor pipeline performance, troubleshoot issues, and improve system reliability.
• Support data quality, validation, documentation, and data standards.

Skills

Python

SQL

Bash

Spark

Flink

Hadoop ecosystem

Kafka

Flume

Java

Scala

Education

Bachelor’s degree in Computer Science, Engineering, or a related field

Tools

HDFS

Hive

Job description

A leading technology company in Singapore is seeking a skilled Data Engineer to design, develop, and maintain data pipelines. The ideal candidate will have strong proficiency in Python, SQL, and Bash, along with hands-on experience in frameworks like Spark and Flink. Responsibilities include building large-scale data processing workflows and collaborating with cross-functional teams to meet business needs. A Bachelor's degree in Computer Science or a related field is required, along with experience in the AdTech industry.