Senior Data Engineer: Build & Optimize Scalable Pipelines

Capgemini Hong Kong Limited; Hong Kong; On-site; HKD 480,000 - 760,000; Full time

Job summary

Capgemini Hong Kong Limited is seeking a data engineer to build and maintain batch and streaming data pipelines end-to-end. You will ingest, transform, validate and publish data, and develop reusable transformation patterns for analytics and operations.

The role requires hands-on Python/Java, SQL, Spark, Airflow, and cloud experience, with an emphasis on data quality, performance optimization, and collaboration with upstream/downstream teams.

Qualifications

• 5+ years of data engineering delivery experience.
• Strong Python/Java with advanced SQL and data modeling.
• Experience with Spark and orchestration like Airflow.
• Experience with cloud data platforms (Azure/AWS/GCP).
• Familiar with streaming platforms and data governance basics.

Responsibilities

• Build end-to-end batch and streaming pipelines.
• Develop reusable data transformation patterns.
• Design, test, deploy, and support from requirements to production.
• Implement data quality checks and monitoring.
• Collaborate with upstream/downstream teams.
• Maintain documentation and runbooks.
• Troubleshoot production issues and perform root-cause analysis.

Skills

Python

Java

SQL

Data modelling

Spark

Airflow

Kafka

Git

CI/CD

Observability

Azure/AWS/GCP

Tools

Spark

Airflow

Kafka

Git

CI/CD

Azure

AWS

GCP

Job description

Capgemini Hong Kong Limited is seeking a data engineer to build and maintain batch and streaming data pipelines end-to-end. You will ingest, transform, validate and publish data, and develop reusable transformation patterns for analytics and operations.

The role requires hands-on Python/Java, SQL, Spark, Airflow, and cloud experience, with an emphasis on data quality, performance optimization, and collaboration with upstream/downstream teams.