Software Engineer (Big Data) - Application Computing

Ellis Technologies, Inc.; Singapore; On-site; SGD 80,000 - 120,000; Full time

Job summary

Ellis Technologies, Inc. is looking for an experienced professional to join TikTok's Recommendation Architecture Team in Singapore. This role focuses on designing and implementing real-time data systems, ensuring system stability, and developing industry-leading streaming computing frameworks.

The ideal candidate will have a strong programming background in Java, C++, Scala, and Python, with extensive experience in streaming computing systems and data lake technologies. Join a diverse team dedicated to inspiring creativity and driving technological innovation at TikTok.

Qualifications

• 5+ years of relevant experience in developing real-time computing systems.
• Deep understanding of streaming computing systems.
• Experience in data lake technologies such as Hudi or DeltaLake.

Responsibilities

• Design and implement real-time data systems for large-scale recommendations.
• Troubleshoot production system failures and ensure system stability.
• Research and develop software solutions based on user needs.

Skills

Java

C++

Scala

Python

Streaming computing systems

Flink

Tools

Hadoop

Spark

Kubernetes

Job description

Overview

Team Introduction: TikTok’s Recommendation Architecture Team is responsible for real-time computing direction, handling the design and development of real-time computing systems for TikTok videos, live streams, e-commerce, and a billion-user product recommendation system. Their main focus is ensuring system stability and high availability. They abstract general real-time computing systems, build a unified recommendation feature middleware, and implement a flexible and scalable high-performance storage system and computing model to enable advanced real-time data systems for deduplication, counting, feature services, and other recommendation-related business needs.

Responsibilities

• Design and implement real-time (streaming computing) data systems for large-scale recommendation systems.
• Create flexible, scalable, stable, and high-performance storage systems and computing models.
• Troubleshoot production system failures, design and implement necessary mechanisms and tools to ensure overall stability of the production systems.
• Construct industry-leading streaming computing frameworks and other distributed systems to provide reliable infrastructure for massive data and large-scale business systems.
• Research, design, and develop computer and network software or specialised utility programs.
• Analyze user needs and develop software solutions, applying principles and techniques of computer science, engineering, and mathematical analysis.
• Update software, enhance existing software capabilities, and develop and direct software testing and validation procedures.
• Work with computer hardware engineers to integrate hardware and software systems and develop specifications and performance requirements.

Minimum Qualifications

• Proficient in programming languages like Java, C++, Scala, Python.
• Strong coding and troubleshooting skills.
• At least 5 years of relevant experience.
• Deep understanding of streaming computing systems, with formal production experience in developing TB-level Flink real-time computing systems. Proficient in modules like FlinkDataStream, FlinkSQL, FlinkCheckpoint, FlinkState, and preferably with experience in reading Flink source code.
• Experience in data lake development is preferred. Familiar with at least one data lake technology such as Hudi, Iceberg, DeltaLake, and preferably with experience in reading their source code.
• Willingness to tackle problems without clear answers, with a strong passion for learning new technologies.

Preferred Qualifications

• Experience in handling PB-level data is a plus.
• Familiarity with other big data systems is preferred, including YARN, K8S, Spark, SparkSQL, Kudu, and others.
• Experience in storage systems such as HBase, Cassandra, RocksDB.

Job Information

About TikTok: TikTok is the leading destination for short-form mobile video. Our mission is to inspire creativity and bring joy. TikTok's global headquarters are in Los Angeles and Singapore, with offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo.

Why Join Us

Inspiring creativity is at the core of TikTok's mission. Our innovative product helps people authentically express themselves, discover and connect. We work with global, diverse teams to create value for communities, inspire creativity, and bring joy.

Diversity & Inclusion

TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people across the globe, and our workplace reflects the many communities we reach. We are passionate about celebrating diverse voices and creating an environment that reflects them.

Ready for a career at TikTok?

Discover a career that energizes and excites you every day.