Data Engineer (Geospatial Architecture & Infrastructure)

LyncLab

Kuala Lumpur

On-site

MYR 120,000 - 180,000

Full time

20 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

LyncLab is seeking a Data Engineer to design, build and run the data platform for maritime products, from edge receivers to streaming platforms, storage and analytics. The role is hands-on and includes on-site server setup and integration with client networks.

You will work across data ingestion, processing, warehouse design and governance, while collaborating with data scientists and product teams. A strong background in streaming, SQL, and geospatial data is essential for success.

Qualifications

  • Degree in Computer Science, Software or Data Engineering, or equivalent experience.
  • 5+ years building and operating production data pipelines, including streaming.
  • Fluent in Python, plus at least one of Java, Scala, Rust or Go.
  • Advanced SQL with hands-on BigQuery and PostgreSQL: schema design, partitioning, indexing, query tuning and cost control.
  • Kafka in production, and a stream-processing framework (Flink preferred, in Java or PyFlink; Kafka Streams, Spark Structured Streaming or Beam also considered).
  • Workflow orchestration with Airflow, cron or similar schedulers.
  • Linux administration and networking (TCP/IP, subnets and VLANs, routing, DNS, firewalls, VPN, TLS); able to take a server from bare metal to a running service.
  • Sound engineering practice: idempotency, replayability, observability, Git, CI/CD, Docker, and data quality checks in production pipelines.
  • Experience with geospatial data and analytics-ready data models.

Responsibilities

  • Build and run publishers that move raw AIS/NMEA streams from edge servers and external feeds into the streaming platform, with local buffering and no silent message loss.
  • Administer the streaming platform: topic design, retention, authentication and access control.
  • Develop and operate stream-processing jobs for decoding, enrichment, and event detection.
  • Build incremental, idempotent batch pipelines and run backfills and reprocessing from the raw archive.
  • Measure freshness and latency, tracing missing records to the stage that dropped them.

Skills

Python
Java
Scala
Rust
Go
Advanced SQL

Education

Bachelor's degree in Computer Science, Software or Data Engineering

Tools

BigQuery
PostgreSQL
Kafka
Flink
Airflow
Docker
Kubernetes

Job description

We are looking for a Data Engineer to design, build and run the data platform behind our maritime products: live vessel tracking, marine traffic and risk studies, and the data science and AI systems built on them. You will own the whole path, from the edge server at a receiver site, through stream processing, to the analytical and operational databases, together with the servers and networks it runs on. The role is hands‑on, and part of it is on site: installing servers, connecting them to the network and bringing them up from a bare operating system.

Key Responsibilities
  • Build and run publishers that move raw AIS/NMEA streams from edge servers and external feeds into the streaming platform, with local buffering, reconnect with back‑off, and no silent message loss.
  • Administer the streaming platform: topic, partition and key design, retention, consumer groups and offsets, authentication and access control.
  • Develop and operate stream‑processing jobs for decoding, enrichment, latest‑position state and event detection (anchoring, berthing, zone entry and exit, proximity), handling event time, late data, checkpoints and backpressure.
  • Build incremental, idempotent batch pipelines, and run backfills and reprocessing from the raw archive.
  • Measure freshness, lag and loss at every stage, and trace a missing record to the stage that dropped it.
  • Design raw → staging → curated models for trajectories, reference data (ports, anchorages, zones, boundaries) and spatial outputs (geofencing, proximity, grid aggregates), covering geometry storage, spatial indexing, partitioning, clustering and naming conventions.
  • Plan and execute large‑scale data migrations with change data capture, reconciliation and minimal‑downtime cutover.
Data Warehouse & Databases
  • Write advanced SQL to pull data for analysts, data scientists and client studies, and turn repeated pulls into curated tables or parameterized templates.
  • Manage warehouse datasets (partitioning, clustering, expiration, schema evolution, access control, data catalog), and keep loads complete, deduplicated and reconciled with source counts.
  • Control cost and performance: query cost estimates and limits, materialized and incremental tables, and spend monitoring in the warehouse; indexing, query plans, maintenance, replicas and retention in operational databases.
Infrastructure & Networking
  • Set up servers end to end at client premises, data centers and receiver stations: rack and cable them, connect them to switches, configure IP addressing, VLANs, routing, DNS, NTP and firewalls, and install and harden the operating system.
  • Deploy and support on‑premise installations, working within each client’s security and change‑control rules.
  • Run cloud infrastructure (virtual machines, managed databases, object storage, DNS, CDN and certificates) and containerized services that come back by themselves after a reboot; plan capacity and test backups and restores.
Reliability, Data Quality & Governance
  • Lead CI/CD, infrastructure automation, environment separation and version‑controlled schema migrations.
  • Run monitoring, alerting, SLAs and runbooks; lead incident response and write post‑incident reviews that prevent repeats.
  • Own the data quality framework (automated checks, anomaly detection, reconciliation, quality SLAs), and benchmark data sources against each other on coverage, latency and completeness.
  • Enforce governance: ownership, lineage, access control, retention and secrets management across all data stores.
  • Build curated datasets, knowledge graphs and data interfaces that let agentic AI tools query maritime and geospatial data reliably, with guardrails on quality, lineage and access.
  • Partner with data scientists on study data and production models, mentor others, and document architecture, tables and runbooks.
Qualifications
Must‑Have
  • Degree in Computer Science, Software or Data Engineering, or equivalent experience.
  • 5+ years building and operating production data pipelines, including streaming.
  • Fluent in Python, plus at least one of Java, Scala, Rust or Go.
  • Advanced SQL with hands‑on BigQuery and PostgreSQL: schema design, partitioning, indexing, query tuning and cost control.
  • Kafka in production, and a stream‑processing framework (Flink preferred, in Java or PyFlink; Kafka Streams, Spark Structured Streaming or Beam also considered).
  • Workflow orchestration with Airflow, cron or similar schedulers.
  • Linux administration and networking (TCP/IP, subnets and VLANs, routing, DNS, firewalls, VPN, TLS); able to take a server from bare metal to a running service.
  • Sound engineering practice: idempotency, replayability, observability, Git, CI/CD, Docker, and data quality checks in production pipelines.
  • Experience with geospatial data and analytics‑ready data models.
  • Willing to travel to client sites and receiver stations; government projects may require security vetting.
Preferred
  • Streaming and storage: Flink state, checkpoint and backpressure tuning; PostGIS, TimescaleDB, H3 and spatial indexing (GiST, BRIN).
  • Domain: AIS/NMEA pipelines, or similar telemetry streams from edge or IoT devices.
  • On premise: managed switches, RAID, out‑of‑band management (IPMI, iDRAC, iLO), and government or regulated networks.
  • Platform: Terraform or Ansible, Kubernetes, Prometheus/Grafana, GCP (Cloud Run, Dataflow, Composer), CDC (Debezium, Kafka Connect), Keycloak/OIDC.
  • AI: knowledge graphs and ontology design; AI‑assisted development (e.g. Claude, Codex) in a production workflow.
What We Look For
  • Owners, who treat an outage or a data gap as theirs to find, fix and prevent.
  • Comfortable moving between a server rack, a Kafka topic and a SQL query.
  • Architecture‑first: clear conventions, and a sound balance of cost, performance and maintainability.
  • Growth‑minded and proactive in raising engineering standards in a fast‑moving team.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Scientist (Maritime Analytics & Remote Sensing)
Data Scientist (Maritime Analytics & Remote Sensing)

LyncLab • Kuala Lumpur

On-site
MYR 120,000 - 180,000
Data Engineer
Data Engineer

IPinfo • Batu Pahat

On-site
MYR 180,000 - 280,000
Senior Data Engineer
Senior Data Engineer

Alliance Bank Malaysia Berhad • Kuala Lumpur

On-site
MYR 120,000 - 180,000
Senior Data Engineer(Snowflake / PostgreSQL)
Senior Data Engineer(Snowflake / PostgreSQL)

北京汉克时代科技有限公司 • Kuala Lumpur

On-site
MYR 120,000 - 180,000
Data & Analytics Lead
Data & Analytics Lead

Involve Asia Technologies Sdn Bhd • Kuala Lumpur

On-site
MYR 240,000 - 360,000
Data Engineering (Azure Databricks, Power BI, Python)
Data Engineering (Azure Databricks, Power BI, Python)

Mercedes-Benz Group • Kuala Lumpur

Remote
MYR 180,000 - 250,000
Data Platform Integration Engineer
Data Platform Integration Engineer

北京汉克时代科技有限公司 • Kuala Lumpur

On-site
MYR 120,000 - 180,000
Senior Engineer - Data Engineering
Senior Engineer - Data Engineering

Techstreet Malaysia • Kuala Lumpur

On-site
MYR 120,000 - 180,000
Junior, Data Engineer
Junior, Data Engineer

Tranglo • Kuala Lumpur

On-site
MYR 90,000 - 120,000
Senior Data Engineer
Senior Data Engineer

PERSOL • Kuala Lumpur

On-site
MYR 120,000 - 180,000