Software Engineer (Big Data) , Recommendation Ecosystem Infrastructure (ByteDance Singapore)

BYTEDANCE PTE. LTD.

Singapore

On-site

SGD 120,000 - 180,000

Full time

3 days ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

ByteDance seeks engineers to craft and deploy real-time data systems for a billion-user recommendation platform. You will design scalable storage and compute models, troubleshoot production failures, and build distributed streaming frameworks to support massive data needs in the ads, search and recommendation domains.

You will analyze user needs, apply rigorous CS/engineering methods, and drive software updates, testing and validation.

Qualifications

Proficient in Java, C++, Scala, Python.
Strong coding and troubleshooting skills.
At least 3 years of relevant experience.
Deep understanding of streaming computing systems with production TB-level Flink.
Familiar with Flink modules and data lake tech (Hudi/Iceberg/DeltaLake).
Willing to learn and tackle hard problems.

Responsibilities

Design and implement real-time data systems for large-scale recommendations.
Create scalable, stable storage and compute models.
Troubleshoot production failures; ensure production stability.
Build streaming frameworks and distributed infrastructure for massive data.
Research, design and develop software and utilities.
Analyze user needs and craft software solutions using CS/engineering methods.
Update software and drive testing/validation procedures.
Collaborate with hardware teams to integrate HW/SW and set specs.

Skills

Java

C++

Scala

Python

3+ years exp

Flink streaming

FlinkDataStream

FlinkSQL

FlinkCheckpoint

FlinkState

Data lake

Hudi

Iceberg

DeltaLake

YARN

Kubernetes

Spark

Kudu

RocksDB

Reading code

Tools

FlinkDataStream

FlinkSQL

FlinkCheckpoint

FlinkState

Hudi

Iceberg

DeltaLake

YARN

Kubernetes

Spark

Kudu

RocksDB

Job description

About Us

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Lemon8, CapCut and Pico as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Why Join ByteDance

Inspiring creativity is at the core of ByteDance's mission. Our innovative products are built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and enrich life - a mission we work towards every day.

As ByteDancers, we strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. By constantly iterating and fostering an \"Always Day 1\" mindset, we achieve meaningful breakthroughs for ourselves, our Company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Job highlights

Positive team atmosphere, Career growth opportunity, Flat organization

Responsibilities

Team Introduction

The Data Ecosystem Team has the vital role of crafting and implementing a storage solution for offline data in our recommendation system, which caters to more than a billion users. Their primary objectives are to guarantee system reliability, uninterrupted service, and seamless performance. They aim to create a storage and computing infrastructure that can adapt to various data sources within the recommendation system, accommodating diverse storage needs. Their ultimate goal is to deliver efficient, affordable data storage with easy-to-use data management tools for the recommendation, search, and advertising functions.

Responsibilities:

Design and implement real-time (streaming computing) data systems for large-scale recommendation systems.

Create flexible, scalable, stable, and high-performance storage systems and computing models.

Troubleshoot production system failures, design and implement necessary mechanisms and tools to ensure overall stability of the production systems

Construct industry-leading streaming computing frameworks and other distributed systems to provide reliable infrastructure for massive data and large-scale business systems

Research, design, and develop computer and network software or specialised utility programs.

Analyse user needs and develop software solutions, applying principles and techniques of computer science, engineering, and mathematical analysis.

Update software, enhances existing software capabilities, and develops and direct software testing and validation procedures.

Work with computer hardware engineers to integrate hardware and software systems and develop specifications and performance requirements.

Qualifications

Minimum Qualifications

Proficient in programming languages like Java, C++, Scala, Python.

Strong coding and troubleshooting skills.

At least 3 years of relevant experience

Deep understanding of streaming computing systems, with formal production experience in developing TB-level Flink real-time computing systems. Proficient in modules like FlinkDataStream, FlinkSQL, FlinkCheckpoint, FlinkState, and preferably with experience in reading Flink source code.

Experience in data lake development is preferred. Familiar with at least one data lake technology such as Hudi, Iceberg, DeltaLake, and preferably with experience in reading their source code.