Real-Time Big Data Platform Lead (Contract)

QUESS SINGAPORE; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

QUESS SINGAPORE seeks an experienced data engineering leader to architect and operate scalable data warehouses and both batch and real-time pipelines (Flink streaming, Hive/HDFS batch).

You will own a multi-cluster data platform, governance, monitoring and reliability improvements, while mentoring engineers and collaborating with product and business teams to deliver dependable data solutions.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8+ years in software or data engineering, 4+ years on big data platforms.
• Hands-on production experience with Flink for real-time processing.
• Experience designing multi-tenant data platforms with access control and audit.
• Proficient Java (Spring) with knowledge of Python/Scala/Go a bonus.
• Strong SQL skills; experience with MySQL/Tidb or similar.
• Experience building real-time data services where latency matters.
• Knowledge of data modelling, ETL/ELT, batch and streaming architectures.
• Ability to evaluate trade-offs and mentor engineers.

Responsibilities

• Lead architecture and development of scalable data warehouses and data pipelines (Flink streaming, Hive/HDFS batch).
• Own design and operation of a big data development platform with multi-cluster management (YARN).
• Build real-time user-facing data services (transactions, billing, analytics).
• Design data ingestion systems with redundancy, heartbeat and quotas.
• Define data models, tagging systems and feature datasets for analytics.
• Establish data quality, observability and governance standards and dashboards.
• Plan fault drills and performance tests to reduce risk and boost throughput.
• Improve processing performance and platform stability; scale wisely.
• Provide technical guidance and code reviews; mentor engineers.
• Collaborate with business/product/engineering to translate requirements into data solutions.

Skills

Flink

Hive/HDFS

YARN

Java/Spring

SQL

Python/Scala/Go

Data modelling

ETL/ELT

Monitoring

Distributed systems

Education

Bachelor’s degree in Computer Science or related field

Tools

Spark

Kafka

Trino

Redis

Job description

QUESS SINGAPORE seeks an experienced data engineering leader to architect and operate scalable data warehouses and both batch and real-time pipelines (Flink streaming, Hive/HDFS batch).

You will own a multi-cluster data platform, governance, monitoring and reliability improvements, while mentoring engineers and collaborating with product and business teams to deliver dependable data solutions.