Data Engineer - Traffic Mart

Shopee; Singapore; On-site; SGD 110,000 - 150,000; Full time

Job summary

Shopee Singapore is seeking a seasoned Data Engineer to build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers.

You will collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and AI/ML data stacks.

Qualifications

• Bachelor's degree in Computer Science or related field.
• At least 3 years of data engineering experience.
• Traffic Mart experience is advantageous.
• Experience with Spark, Flink, Hadoop or HBase.
• Proficiency in Java, Scala, Python, SQL.
• Familiar with data warehouse architecture and principles.

Responsibilities

• Build and operate the Marketplace data warehouse end to end.
• Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures.
• Design and optimize large-scale data processing pipelines for offline and streaming platforms.
• Lead data quality and governance efforts, including lineage tracking and SLA monitoring.
• Evaluate emerging data technologies to evolve the platform roadmap.

Skills

Data warehousing

Data governance

Batch and real-time processing

Semantic layer design

Collaboration

Education

Bachelor's Degree in Computer Science or related field

Tools

Spark

Flink

Hadoop

HBase

Java

Scala

Python

SQL

Job description

• Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers.
• Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and algorithmic applications.
• Design and optimize large-scale data processing pipelines, improving throughput, latency, and cost efficiency across offline and streaming platforms.
• Lead data quality and governance efforts, including schema standardization, lineage tracking, SLA monitoring, and best practices that increase reliability and user trust.
• Evaluate and adopt emerging data technologies such as streaming engines, lakehouse architectures, and AI/ML data stacks to address key Marketplace challenges and evolve the platform roadmap.

Job Description

• Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers.
• Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and algorithmic applications.
• Design and optimize large-scale data processing pipelines, improving throughput, latency, and cost efficiency across offline and streaming platforms.
• Lead data quality and governance efforts, including schema standardization, lineage tracking, SLA monitoring, and best practices that increase reliability and user trust.
• Evaluate and adopt emerging data technologies such as streaming engines, lakehouse architectures, and AI/ML data stacks to address key Marketplace challenges and evolve the platform roadmap.

Requirements

• Bachelor's Degree or above in Computer Science or related fields
• At least 3 years of data engineering experience
• Prior experience or knowledge working with Traffic Mart would be advantageous
• Experience with big data processing technologies such as Spark, Flink, Hadoop or HBase
• Proficiency in one or more programming languages, such as Java, Scala, Python, SQL, etc.
• Familiar with data warehouse architecture and principles