Senior Data & Analytics Tech Lead

ZAFIRO CONSULTING PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Zafiro Consulting is seeking a senior data engineering professional with 11–13 years of experience to work on large-scale data pipelines using Spark, Flink, Kafka and NOSQL databases in Singapore. You will leverage Spark Dataframe API and Hive, along with shell scripting, CI/CD processes and cloud infrastructure on Azure or AWS.

Ideal candidates have deep experience across multiple databases and big data tools, and are comfortable operating in a fast-paced environment requiring robust data

Qualifications

• Experience with at least two of the listed DB systems (SQL Server, Oracle, DB2, Netezza).
• Strong experience with Apache Spark Streaming or Apache Flink.
• Hands-on Kafka experience.
• Proficiency with NOSQL databases such as Cosmos DB or Document DB.
• Familiarity with Spark Dataframe API and Hive (HQL).
• Shell or bash scripting ability.
• CI/CD practices and tooling.
• Experience with at least one Cloud Infra provider (Azure or AWS).

Skills

Apache Spark Streaming

Kafka

NOSQL databases

Spark Dataframe API

Hive (HQL)

Shell scripting

CI CD

Cloud Infra (Azure/AWS)

DB knowledge (2 of 4)

Tools

SQL Server / Oracle / DB2 / Netezza

Apache Spark Streaming / Flink

Kafka

Cosmos DB

Document DB

Spark Dataframe API

Hive

Shell

CI/CD

Azure/AWS

Job description

Zafiro Consulting is seeking a senior data engineering professional with 11–13 years of experience to work on large-scale data pipelines using Spark, Flink, Kafka and NOSQL databases in Singapore. You will leverage Spark Dataframe API and Hive, along with shell scripting, CI/CD processes and cloud infrastructure on Azure or AWS.

Ideal candidates have deep experience across multiple databases and big data tools, and are comfortable operating in a fast-paced environment requiring robust data