Senior Big Data Storage & Compute Engineer—Paimon + Flink

TikTok Inc.; San Jose (CA); On-site; USD 218,000 - 480,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401(k) with company match

Parental Leave

Disability Insurance

Wellbeing Benefits

Paid Holidays

Paid Sick Days

Paid Personal Time

Job summary

TikTok Inc. is seeking a data storage and distributed systems engineer to design offline and real-time architectures for large-scale recommendation, search, and advertising platforms.

You will work on Paimon and Flink-based storage foundations powering streaming and batch pipelines for over a billion users. Join a team focused on reliability, scalability, and cost-effective data management, building robust infrastructure to adapt to diverse data sources and evolving business needs while

Qualifications

• A bachelor's degree or above in computer science, software engineering, or related fields, with experience in building scalable systems.
• Thorough understanding of Paimon and Flink, and be able to understand and use them at the source-code level.
• In-depth understanding of at least one data lake technology (such as Paimon), with practical implementation and customization experience, which should be highlighted in the resume.
• Proficiency in programming languages such as Java, C++, and Scala, with strong coding and problem-solving abilities.
• Experience in data warehouse modeling and be able to design efficient data models that meet complex business scenarios.
• Experience in using other big-data systems/frameworks (such as Hive, HBase, Kudu, etc.) and handling large-scale data (PB - level and above).
• Familiar with the principles of HDFS, and knowledge of columnar storage formats such as Parquet and ORC is preferred.

Responsibilities

• Architecture Design and Implementation: Design and implement offline and real-time data architectures for large-scale recommendation, search, and advertising systems based on Paimon and Flink.
• System Construction and Optimization: Design and implement flexible, scalable, stable, and high-performance storage systems and computing models.
• Troubleshooting and Stability Assurance: Be responsible for troubleshooting production systems and ensuring overall stability of the production environment.
• Distributed System Construction: Build industry-leading distributed systems with offline/online storage on Paimon and batch/stream processing with Flink.

Skills

Java

C++

Scala

Education

Bachelor's degree in computer science / software engineering or related field

Tools

Paimon

Flink

Job description

TikTok Inc. is seeking a data storage and distributed systems engineer to design offline and real-time architectures for large-scale recommendation, search, and advertising platforms.

You will work on Paimon and Flink-based storage foundations powering streaming and batch pipelines for over a billion users. Join a team focused on reliability, scalability, and cost-effective data management, building robust infrastructure to adapt to diverse data sources and evolving business needs while