Senior Data Engineer - Data Mart

Shopee; Indonesia; On-site; IDR 360,000,000 - 700,000,000; Full time

Job summary

Shopee is seeking a data engineer to participate in building and enhancing our Marketplace data warehouse, including offline and real-time data pipelines. You will collaborate with multiple teams to design data architectures and deliver efficient data solutions for BI analysis, data products, and algorithm applications.

You will also focus on evolving the data architecture, improving data quality, and handling massive data processing with new technologies to boost productivity.

Qualifications

• Bachelor degree or above in computer related field with at least 3 years of full-time experience.
• Experience with Spark, Flink, Hadoop, HBase, Kafka, Druid or ClickHouse.
• Proficient in Java, Scala, Python or SQL.
• Familiar with data warehouse architecture and performance tuning.
• Strong communication and project coordination skills.

Responsibilities

• Develop data warehouse components across offline and real-time warehouses.
• Support data applications for BI, data products, and algorithms; collaborate with teams.
• Optimize data architecture and ensure data quality and productivity for massive data processing.

Skills

Big data processing

Programming languages

Data warehouse design

Communication skills

Education

Bachelor degree in computer science or related field

Tools

Spark

Flink

Hadoop

HBase

Kafka

Druid

ClickHouse

Job description

Job Description:

• Participate in the development work related to the Marketplace data warehouse, including data collection in offline and real-time data warehouses, data public layer construction, data application layer construction, and data governance.
• Participate in supporting the data application of various business modules, cooperate with different teams to communicate the requirement, and design data architecture to provide data users with efficient data solutions, including supporting various scenarios such as BI analysis, data products, and algorithm applications.
• Participate in the exploration and breakthrough of Marketplace's key data technologies, and be responsible for optimizing and improving the existing data architecture, improving data quality and productivity, and including massive data processing, real-time data processing, and the application of various new technologies.

Requirements:

• Bachelor degree or above in computer related field with at least 3 years of full time working experience.
• Familiarity with one or more big data processing technologies such as Spark, Flink, Hadoop, HBase, Kafka, Druid, Clickhouse, etc.
• Proficiency in one or more programming languages, such as Java, Scala, Python, SQL, etc.
• Familiar with data warehouse architecture and principles, with relevant experience in big data architecture design, model design and performance tuning.
• Good thinking logic and communication skills, as well as good project management and coordination skills