Data Engineer — Python, Spark on AWS & MLOps

Michael Page International (Hong Kong) Limited; Hong Kong; On-site; HKD 446,000 - 781,000; Full time

Benefits offered by this job

Competitive monthly salary

Employer benefits

Job summary

Michael Page International (Hong Kong) Limited is seeking a data engineer to design, develop, and maintain scalable data pipelines on a cloud-based platform built with AWS and Spark.

Ideal candidates will have 3-5 years of hands-on data engineering experience, strong Python/SQL skills, and working knowledge of AWS data services such as S3, Glue, and Redshift. Cantonese and English fluency are required, with Mandarin a bonus.

Qualifications

• 3-5 years of hands-on experience in data engineering.
• Strong practical experience with Apache Spark running on AWS EMR.
• Experience working with AWS data services, including S3, Glue, Athena, Redshift, Step Functions, and Lambda; exposure to Lake Formation and SageMaker is advantageous.
• Proficiency in Python and advanced SQL, with experience in ETL/ELT development and data modelling.
• Familiarity with big data and streaming technologies such as Hive, Presto, Kafka, and Spark Streaming.
• Knowledge of MLOps principles and experience supporting machine learning models in production environments is beneficial.
• Exposure to AI and Generative AI-related projects is an advantage.
• Strong collaboration skills and effective communication abilities.
• Fluent in Cantonese and English; Mandarin proficiency is a plus.
• Prior experience gained within an IT consulting environment or technology services organisation is preferred.

Responsibilities

• Design, develop, and maintain scalable data pipelines and data products within a cloud-based data platform.
• Build, enhance, and troubleshoot Apache Spark workloads on AWS EMR for both batch and real-time data processing.
• Enable self-service analytics and data science capabilities through well-structured, governed, and high-quality datasets.
• Contribute to MLOps processes, including support for model deployment, monitoring, and data preparation activities for production ML solutions.
• Assist with AI and Generative AI initiatives by delivering the necessary data foundations and platform integrations.
• Maintain data quality, governance, lineage, security, and consistency across the end-to-end data lifecycle.
• Collaborate with business users, data specialists, and platform engineering teams to gather requirements and implement data-driven solutions.
• Adhere to established architecture guidelines, engineering best practices, and development standards.

Skills

Spark

Python

SQL

ETL/ELT

Data modelling

MLOps

Data governance

Kafka

Spark Streaming

Hive

Presto

Cloud data platforms

Tools

AWS EMR

S3

Glue

Athena

Redshift

Step Functions

Lambda

Lake Formation

SageMaker

Job description

Michael Page International (Hong Kong) Limited is seeking a data engineer to design, develop, and maintain scalable data pipelines on a cloud-based platform built with AWS and Spark.

Ideal candidates will have 3-5 years of hands-on data engineering experience, strong Python/SQL skills, and working knowledge of AWS data services such as S3, Glue, and Redshift. Cantonese and English fluency are required, with Mandarin a bonus.