Big Data Engineer, Data Ingestion

BYTEDANCE PTE. LTD.; Singapore; On-site; SGD 90,000 - 130,000; Full time

Benefits offered by this job

Positive team atmosphere

Career growth opportunity

Paid leave

Industry experts

Meals provided

Job summary

ByteDance Singapore is seeking a software engineer for the Data Management Suite team to build and optimize architecture for large-scale data ingestion systems, enabling real-time pipelines and scalable data cataloging.

You will work with cross-functional teams to improve performance and reliability, and contribute to a cutting-edge data platform used across the company. This role offers growth in a fast-paced tech environment.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 2+ years in software development.
• Experience with big data tech stacks required.

Responsibilities

• Design, develop, and optimize data ingestion systems for real-time pipelines.
• Improve performance, scalability, and reliability of data ingestion
• Collaborate with cross-functional teams to deliver event data ingestion and storage solutions.
• Troubleshoot and resolve production issues to minimize downtime.

Skills

Java

Python

Distributed systems

Big data

Education

Bachelor's degree in Computer Science or equivalent

Tools

Hadoop

Flink

Kafka

Go

Job description

About Us

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Lemon8, CapCut and Pico as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Why Join ByteDance

Inspiring creativity is at the core of ByteDance's mission. Our innovative products are built to help people authentically express themselves, discover and connect - and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and enrich life - a mission we work towards every day.

As ByteDancers, we strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our Company, and our users. When we create and grow together, the possibilities are limitless. Join us.

Diversity & Inclusion

ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.

Job highlights

• Positive team atmosphere
• Career growth opportunity
• Paid leave
• Industry experts
• Meals provided

Responsibilities

Team Introduction

The Data Management Suite team is building products that cover the whole lifecycle of data pipeline, including data ingestion and Integration, data development, data catalog, data security and data governance. These products support various businesses, so data engineers and data scientists could greatly boost their productivity.As a software engineer in the data management suite team, you will have the opportunity to build, optimize and grow one of the largest data platforms in the world. You'll have the opportunity to gain hands-on experience on core systems in the data platform ecosystem. Your work will have a direct and huge impact on the company's core products as well as hundreds of millions of users.

Responsibilities:

• Design, develop, and optimize the architecture of large-scale data ingestion systems to support real-time data pipelines, ensuring high throughput, low latency, and fault tolerance.
• Enhance the performance, scalability, and reliability of data ingestion pipelines.
• Develop and implement automated and intelligent operation and maintenance systems to monitor, diagnose, and ensure the stability and reliability of data ingestion pipelines.
• Collaborate with cross-functional teams to deliver event data ingestion, transformation, and storage solutions that meet diverse business requirements.
• Troubleshoot and resolve complex issues in production systems, ensuring minimal downtime and optimal performance.

Qualifications

Minimum Qualifications:

• Bachelor's degree in Computer Science or equivalent practical experience.
• 2+ years of experience in software development with proficiency in one or more programming languages, such as Java, Scala, Python, or Go.
• Strong understanding of data structures, algorithms, and distributed systems principles.
• Hands-on experience with big data technologies, such as Hadoop, Flink, Kafka, or similar frameworks.

Preferred Qualifications:

• 2+ years of experience in real-time data processing frameworks like Flink. Contributions to open-source projects are a plus.
• Experience in the event tracking domain, including event collection, real-time processing, governance, quality assurance and cost optimization.
• Deep understanding of data lakehouse architectures and the integration of stream and batch processing.
• Strong problem-solving skills and the ability to work in a collaborative environment.