Real-Time Data Platform Engineer

北京汉克时代科技有限公司; Kuala Lumpur; On-site; MYR 120,000 - 180,000; Full time

Job summary

北京汉克时代科技有限公司 is seeking an experienced Data Platform Engineer for the Kuala Lumpur team. You will maintain core data pipelines across NiFi, Kafka, Snowflake, and PostgreSQL, ensuring robust ingestion and real-time processing.

The role requires collaboration with overseas teams and a fully English working environment. You will handle incident response, capacity planning, and production deployments, using APM and log-monitoring tools to detect issues early and drive resolution.

Qualifications

• Bachelor's degree or above in Computer Science, Software Engineering, Information Management or related discipline.
• Around three years of experience in data platforms, data integration, or real-time data development.
• Hands-on Apache NiFi experience including ingestion, flow configuration and troubleshooting.
• Experience with Kafka or Azure Event Hub and understanding of messaging concepts.
• Experience with Snowflake data warehouse, loading and basic optimisation.
• Proficiency with PostgreSQL and database performance troubleshooting.
• Proficiency with Linux and using logs/CLI for issue location.
• Experience with monitoring tools like Splunk or Grafana.
• Familiar with Airflow and Azure Functions orchestration.
• Strong ownership, cross-team collaboration and ability to work under pressure.

Responsibilities

• Join the data platform operations team to maintain and optimise core integration pipelines.
• Maintain NiFi, Kafka and Snowflake ingestion pipelines for reliable data flows.
• Maintain Event Hub, Flink and PostgreSQL real-time processing pipelines.
• Manage daily operations, capacity assessment and incident handling for core tools.
• Collaborate with upstream/downstream teams on data onboarding and quality checks.
• Support production changes, releases, on-call response and incident handling.
• Use APM and log-monitoring tools to track pipeline health and drive issue resolution.
• Prepare English technical docs, operation manuals and handover materials.

Skills

NiFi

Kafka

Event Hub

Snowflake

PostgreSQL

Linux

Grafana

Airflow

Azure Functions

APM / logs

Cross-team collaboration

Education

Bachelor's degree or above in CS/SE/Info Mgmt

Tools

Splunk

Grafana

Airflow

Azure Functions

Job description

北京汉克时代科技有限公司 is seeking an experienced Data Platform Engineer for the Kuala Lumpur team. You will maintain core data pipelines across NiFi, Kafka, Snowflake, and PostgreSQL, ensuring robust ingestion and real-time processing.

The role requires collaboration with overseas teams and a fully English working environment. You will handle incident response, capacity planning, and production deployments, using APM and log-monitoring tools to detect issues early and drive resolution.