Data Engineer

Shopee; Jakarta Pusat; On-site; IDR 180,000,000 - 320,000,000; Full time

Job summary

Shopee is seeking a data engineer to design and implement scalable data marts and data warehouses in Jakarta. You will manage end-to-end pipelines, ensuring a single source of truth, and collaborate with Data PMs, BI analysts, and engineering teammates to deliver high-impact insights within project timelines.

The role emphasizes data quality, security, and privacy, with best-practice software development including test-driven development and thorough documentation.

Qualifications

• Hold a Masters or Bachelors degree in Computer Science, Information Systems, or other quantitative-based majors.

Responsibilities

• Design and develop data marts and data warehouses to support business intelligence and data analytics initiatives within the project lifecycle.
• Handle end-to-end data pipelines from multiple internal and external sources, ensuring high performance, scalability, and establishing a Single Source of Truth for project deliverables.
• Collaborate closely with Data PMs, Business Intelligence Analysts, and engineering team members to transform complex data into actionable business insights within designated project timelines.
• Implement and maintain data management best practices across all project pipelines, focusing on data quality, security, and privacy.
• Execute software development best practices, including test-driven development and code documentation, to ensure robust and maintainable data infrastructure.

Skills

SQL

Spark

HDFS

Python

Scala

OO programming

Functional programming

UNIX

Batch processing

Kimball data modeling

Education

Master's or Bachelor's degree in Computer Science, Information Systems, or other quantitative majors

Tools

Version control

Test-driven development

Technical documentation

SDLC

Job description

• Design and develop data marts and data warehouses to support business intelligence and data analytics initiatives within the project lifecycle.
• Handle end-to-end data pipelines from multiple internal and external sources, ensuring high performance, scalability, and establishing a Single Source of Truth for project deliverables.
• Collaborate closely with Data PMs, Business Intelligence Analysts, and engineering team members to transform complex data into actionable business insights within designated project timelines.
• Implement and maintain data management best practices across all project pipelines, focusing on data quality, security, and privacy.
• Execute software development best practices, including test-driven development and code documentation, to ensure robust and maintainable data infrastructure.

Job Description

• Design and develop data marts and data warehouses to support business intelligence and data analytics initiatives within the project lifecycle.
• Handle end-to-end data pipelines from multiple internal and external sources, ensuring high performance, scalability, and establishing a Single Source of Truth for project deliverables.
• Collaborate closely with Data PMs, Business Intelligence Analysts, and engineering team members to transform complex data into actionable business insights within designated project timelines.
• Implement and maintain data management best practices across all project pipelines, focusing on data quality, security, and privacy.
• Execute software development best practices, including test-driven development and code documentation, to ensure robust and maintainable data infrastructure.

Requirements

• Hold a Master\'s or Bachelor\'s degree in Computer Science, Information Systems, or other quantitative-based majors (Science or Engineering)
• Possess 2 to 4 years of hands-on working experience in data engineering, data pipeline development, or data architecture.
• Demonstrated expert-level proficiency in SQL, Spark, HDFS, and big data processing frameworks.
• Expert knowledge of Object-Oriented and Functional Programming, with a strong preference for Python or Scala.
• Solid experience working in UNIX environments, with deep understanding of batch processing, lambda architecture, and Kimball data modeling concepts.
• Proficient in Software Development Life Cycle (SDLC) practices, including version control, test-driven development, and technical documentation.