Big Data Engineer, Data Ingestion

BYTEDANCE PTE. LTD.; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

BYTEDANCE PTE. LTD. invites a data engineer to join a team building the data management suite. You will design, develop, and optimize large-scale data ingestion systems supporting real-time pipelines, with focus on throughput, latency, and fault tolerance.

You will work with Hadoop, Flink, and Kafka to enhance performance, reliability, and monitoring across data platforms used by hundreds of millions of users. A collaborative, cross-functional environment is expected to drive scalable solutions.

Qualifications

• Bachelor's degree in Computer Science or equivalent practical experience.
• 2+ years of software development with Java/Scala/Python/Go proficiency.
• Strong understanding of data structures, algorithms, and distributed systems.
• Hands-on experience with big data technologies such as Hadoop, Flink, Kafka.
• 2+ years real-time data processing experience, e.g., Flink.
• Open-source contributions are a plus.
• Experience in event tracking, real-time processing, governance and cost optimization.
• Deep understanding of data lakehouse architectures and stream/batch integration.
• Strong problem-solving skills and collaborative mindset.

Responsibilities

• Design, develop, and optimize large-scale data ingestion systems for real-time pipelines.
• Enhance performance, scalability, and reliability of data ingestion workflows.
• Develop automated operation and maintenance systems to monitor and diagnose pipelines.
• Collaborate with cross-functional teams on event data ingestion, transformation, and storage.
• Troubleshoot production issues to minimize downtime and improve performance.

Skills

Java

Python

Scala

Go

Education

Bachelor's degree in Computer Science

Tools

Hadoop

Flink

Kafka

Job description

Responsibilities

The Data Management Suite team is building products that cover the whole lifecycle of data pipeline, including data ingestion and Integration, data development, data catalog, data security and data governance. These products support various businesses, so data engineers and data scientists could greatly boost their productivity. As a software engineer in the data management suite team, you will have the opportunity to build, optimize and grow one of the largest data platforms in the world.

You’ll have the opportunity to gain hands‑on experience on core systems in the data platform ecosystem. Your work will have a direct and huge impact on the company's core products as well as hundreds of millions of users. Responsibilities:

• Design, develop, and optimize the architecture of large-scale data ingestion systems to support real‑time data pipelines, ensuring high throughput, low latency, and fault tolerance.
• Enhance the performance, scalability, and reliability of data ingestion pipelines.
• Develop and implement automated and intelligent operation and maintenance systems to monitor, diagnose, and ensure the stability and reliability of data ingestion pipelines.
• Collaborate with cross‑functional teams to deliver event data ingestion, transformation, and storage solutions that meet diverse business requirements.
• Troubleshoot and resolve complex issues in production systems, ensuring minimal downtime and optimal performance.

Qualifications

Minimum Qualification(s):

• Bachelor's degree in Computer Science or equivalent practical experience.
• 2+ years of experience in software development with proficiency in one or more programming languages, such as Java, Scala, Python, or

Go.

• Strong understanding of data structures, algorithms, and distributed systems principles.
• Hands‑on experience with big data technologies, such as Hadoop, Flink, Kafka, or similar frameworks.

Preferred Qualification(s):

• 2+ years of experience in real‑time data processing frameworks like Flink.

Contributions to open‑source projects are a plus.

• Experience in the event tracking domain, including event collection, real‑time processing, governance, quality assurance and cost optimization.
• Deep understanding of data lakehouse architectures and the integration of stream and batch processing.
• Strong problem‑solving skills and the ability to work in a collaborative environment.

Job Information

Founded in 2012, ByteDance's mission is to inspire creativity and enrich life. With a suite of more than a dozen products, including TikTok, Lemon8, CapCut and Pico as well as platforms specific to the China market, including Toutiao, Douyin, and Xigua, ByteDance has made it easier and more fun for people to connect with, consume, and create content.

Diversity & Inclusion

ByteDance is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At ByteDance, our mission is to inspire creativity and enrich life. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too.