Real-Time Data Ingestion Engineer for Large-Scale Pipelines

Pangle; Singapore; On-site; SGD 90,000 - 140,000; Full time

Job summary

ByteDance in Singapore is seeking a Software Engineer for the Data Management Suite to design and optimize large-scale data ingestion systems, enabling real-time pipelines and fault tolerance.

You will work with cross-functional teams to deliver event data ingestion, transformation, and storage solutions, applying knowledge of big data tech and distributed systems to impact hundreds of millions of users.

Qualifications

• Bachelor's degree in Computer Science or equivalent practical experience.
• 2+ years of experience in software development with proficiency in one or more programming languages (Java/Scala/Python/Go).
• Strong understanding of data structures, algorithms, and distributed systems principles.
• Hands-on experience with big data technologies such as Hadoop, Flink, Kafka, or similar frameworks.

Responsibilities

• Design, develop, and optimize the architecture of large-scale data ingestion systems to support real-time data pipelines, ensuring high throughput, low latency, and fault tolerance.
• Enhance the performance, scalability, and reliability of data ingestion pipelines.
• Develop and implement automated and intelligent operation and maintenance systems to monitor, diagnose, and ensure the stability and reliability of data ingestion pipelines.
• Collaborate with cross-functional teams to deliver event data ingestion, transformation, and storage solutions that meet diverse business requirements.
• Troubleshoot and resolve complex issues in production systems, ensuring minimal downtime and optimal performance.

Skills

Java

Scala

Python

Go

Distributed systems

Education

Bachelor's degree in Computer Science

Tools

Hadoop

Flink

Kafka

Job description

ByteDance in Singapore is seeking a Software Engineer for the Data Management Suite to design and optimize large-scale data ingestion systems, enabling real-time pipelines and fault tolerance.

You will work with cross-functional teams to deliver event data ingestion, transformation, and storage solutions, applying knowledge of big data tech and distributed systems to impact hundreds of millions of users.