Data Engineer (Python Backend, IT)47k

Michael Page International (Hong Kong) Limited; Hong Kong; On-site; HKD 446,000 - 781,000; Full time

Benefits offered by this job

Competitive monthly salary

Employer benefits

Job summary

Michael Page International (Hong Kong) Limited is seeking a data engineer to design, develop, and maintain scalable data pipelines on a cloud-based platform built with AWS and Spark.

Ideal candidates will have 3-5 years of hands-on data engineering experience, strong Python/SQL skills, and working knowledge of AWS data services such as S3, Glue, and Redshift. Cantonese and English fluency are required, with Mandarin a bonus.

Qualifications

• 3-5 years of hands-on experience in data engineering.
• Strong practical experience with Apache Spark running on AWS EMR.
• Experience working with AWS data services, including S3, Glue, Athena, Redshift, Step Functions, and Lambda; exposure to Lake Formation and SageMaker is advantageous.
• Proficiency in Python and advanced SQL, with experience in ETL/ELT development and data modelling.
• Familiarity with big data and streaming technologies such as Hive, Presto, Kafka, and Spark Streaming.
• Knowledge of MLOps principles and experience supporting machine learning models in production environments is beneficial.
• Exposure to AI and Generative AI-related projects is an advantage.
• Strong collaboration skills and effective communication abilities.
• Fluent in Cantonese and English; Mandarin proficiency is a plus.
• Prior experience gained within an IT consulting environment or technology services organisation is preferred.

Responsibilities

• Design, develop, and maintain scalable data pipelines and data products within a cloud-based data platform.
• Build, enhance, and troubleshoot Apache Spark workloads on AWS EMR for both batch and real-time data processing.
• Enable self-service analytics and data science capabilities through well-structured, governed, and high-quality datasets.
• Contribute to MLOps processes, including support for model deployment, monitoring, and data preparation activities for production ML solutions.
• Assist with AI and Generative AI initiatives by delivering the necessary data foundations and platform integrations.
• Maintain data quality, governance, lineage, security, and consistency across the end-to-end data lifecycle.
• Collaborate with business users, data specialists, and platform engineering teams to gather requirements and implement data-driven solutions.
• Adhere to established architecture guidelines, engineering best practices, and development standards.

Skills

Spark

Python

SQL

ETL/ELT

Data modelling

MLOps

Data governance

Kafka

Spark Streaming

Hive

Presto

Cloud data platforms

Tools

AWS EMR

S3

Glue

Athena

Redshift

Step Functions

Lambda

Lake Formation

SageMaker

Job description

• Opportunity to work on a modern AWS-based data and AI platform.
• Exposure to Spark, MLOps, and emerging AI/GenAI initiatives.

About Our Client

Our client is an organisation investing in modern data platform capabilities to support analytics, machine learning, and AI-driven business initiatives. The environment emphasizes cloud technologies, data governance, and engineering best practices.

Job Description

• Design, develop, and maintain scalable data pipelines and data products within a cloud-based data platform.
• Build, enhance, and troubleshoot Apache Spark workloads on AWS EMR for both batch and real-time data processing.
• Enable self-service analytics and data science capabilities through well-structured, governed, and high-quality datasets.
• Contribute to MLOps processes, including support for model deployment, monitoring, and data preparation activities for production ML solutions.
• Assist with AI and Generative AI initiatives by delivering the necessary data foundations and platform integrations.
• Maintain data quality, governance, lineage, security, and consistency across the end-to-end data lifecycle.
• Collaborate with business users, data specialists, and platform engineering teams to gather requirements and implement data-driven solutions.
• Adhere to established architecture guidelines, engineering best practices, and development standards.

The Successful Applicant

• 3-5 years of hands-on experience in data engineering.
• Strong practical experience with Apache Spark running on AWS EMR.
• Experience working with AWS data services, including S3, Glue, Athena, Redshift, Step Functions, and Lambda; exposure to Lake Formation and SageMaker is advantageous.
• Proficiency in Python and advanced SQL, with experience in ETL/ELT development and data modelling.
• Familiarity with big data and streaming technologies such as Hive, Presto, Kafka, and Spark Streaming.
• Knowledge of MLOps principles and experience supporting machine learning models in production environments is beneficial.
• Exposure to AI and Generative AI-related projects is an advantage.
• Strong collaboration skills and effective communication abilities.
• Fluent in Cantonese and English; Mandarin proficiency is a plus.
• Prior experience gained within an IT consulting environment or technology services organisation is preferred.

What's on Offer

• Competitive monthly salary.
• Access to the usual benefits provided by the employer.