Senior Data Platform Engineer - PB-Scale Spatial Lakehouse

Dada Consultants Pte Ltd; Singapore; On-site; SGD 180,000 - 260,000; Full time

Job summary

Dada Consultants Pte Ltd is hiring a hands-on Senior Data Infrastructure Engineer to build and scale a petabyte-scale data lakehouse powering our spatial data products. You will design storage layouts, implement data versioning, and run large-scale processing pipelines in production.

In this role you will integrate ML training data pipelines, optimize cross-region transfers, and maintain IaC on AWS. You will work with Spark, Ray, Flink, Iceberg/Delta Lake, and LakeFS to deliver robust data

Qualifications

• 6+ years in data infrastructure or distributed systems engineering.
• Proven live petabyte-scale data platform experience with concrete throughput and failure modes.
• Python plus Go/Rust/C++/Java/Scala proficiency.
• Iceberg/Delta Lake and LakeFS experience.
• Spark, Ray or Flink processing; Airflow/Argo/Prefect orchestration.
• AWS (S3, EKS/ECS) and Terraform experience.

Responsibilities

• PB-Scale Lakehouse: design object storage layouts, partitions, catalogs, and open formats (Iceberg/Delta).
• Data Versioning & Ingest: build dataset lineage (LakeFS) and high-throughput ingest pipelines.
• Distributed CPU/GPU Compute: operate Spark, Ray, Flink for batch jobs and SLAM/3D processing.
• ML Training Integration: manage GPU clusters with WebDataset, Mosaic Streaming, and cross-region transfers.
• Curation & Querying: build indexing and fast SQL layers (Trino, DuckDB, Spark SQL) for researchers.
• Reliability & Infrastructure: maintain Terraform IaC on AWS and handle backfills/recovery.

Skills

Python

Go

Rust

C++

Java

Scala

Tools

Iceberg/Delta Lake

LakeFS

Glue/Nessie

Spark

Flink

Airflow

Argo

Prefect

Terraform

AWS S3

EKS/ECS

Kubernetes

Job description

Dada Consultants Pte Ltd is hiring a hands-on Senior Data Infrastructure Engineer to build and scale a petabyte-scale data lakehouse powering our spatial data products. You will design storage layouts, implement data versioning, and run large-scale processing pipelines in production.

In this role you will integrate ML training data pipelines, optimize cross-region transfers, and maintain IaC on AWS. You will work with Spark, Ray, Flink, Iceberg/Delta Lake, and LakeFS to deliver robust data