Senior Big Data Engineer - Real-Time Streaming & Storage

BYTEDANCE PTE. LTD.; Singapore; On-site; SGD 180,000 - 260,000; Full time

Job summary

ByteDance's Data Ecosystem Team in Singapore is seeking a senior software engineer to design and build real-time streaming data systems for our large-scale recommendation platform. You will create flexible, scalable storage and computing models that support TB-level data, ensure reliability, and enable easy data management for the recommendation, search, and advertising functions.

The role requires deep expertise in streaming frameworks (Flink), data lake technologies (Hudi, Iceberg, DeltaLake),

Qualifications

• Proficient in Java, C++, Scala, Python.
• Strong coding and troubleshooting skills.
• At least 5 years of relevant experience.
• Deep understanding of streaming computing systems with production experience in TB-level Flink.
• Experience in data lake development (Hudi, Iceberg, DeltaLake).
• Willingness to tackle problems and learn new technologies.

Responsibilities

• Design and implement real-time (streaming computing) data systems for large-scale recommendation systems.
• Create flexible, scalable, stable, and high-performance storage systems and computing models.
• Troubleshoot production system failures, design and implement mechanisms to ensure overall stability of production systems.
• Construct industry-leading streaming computing frameworks and other distributed systems for reliable infrastructure.
• Research, design, and develop computer and network software or specialised utility programs.
• Analyse user needs and develop software solutions applying computer science and engineering principles.
• Update software, enhance existing capabilities, and develop testing and validation procedures.
• Work with hardware engineers to integrate hardware and software systems and develop performance specifications.

Skills

Java

C++

Scala

Python

Streaming computing

Troubleshooting

Flink

Data lake concepts

Tools

FlinkDataStream

FlinkSQL

FlinkCheckpoint

FlinkState

Hudi

Iceberg

DeltaLake

YARN

K8S

Spark

Kudu

RocksDB

Job description

ByteDance's Data Ecosystem Team in Singapore is seeking a senior software engineer to design and build real-time streaming data systems for our large-scale recommendation platform. You will create flexible, scalable storage and computing models that support TB-level data, ensure reliability, and enable easy data management for the recommendation, search, and advertising functions.

The role requires deep expertise in streaming frameworks (Flink), data lake technologies (Hudi, Iceberg, DeltaLake),