Data Engineering Lead: Spark on AWS + Real-Time Data

Michael Page International (Hong Kong) Limited; Hong Kong; On-site; HKD 781,000 - 1,451,000; Full time

Benefits offered by this job

Competitive salary

Contractor benefits

Job summary

Michael Page International (Hong Kong) Limited seeks a Data Engineer Lead to guide a data engineering team and shape scalable data platforms. You will drive technical direction, review designs, and ensure delivery excellence across Spark on AWS EMR and streaming APIs.

You will promote MLOps, collaborate with data science teams, and contribute to governance, security, and cost-efficient solutions while mentoring engineers and engaging stakeholders.

Qualifications

• Bachelor's degree or equivalent technical experience.
• 6+ years in data engineering with leadership experience.
• Ability to present complex tech to varied audiences.
• Proven Spark on AWS EMR, including performance optimization and cost management.
• Experience designing streaming API services for real-time data.
• Strong AWS data services knowledge (S3, Glue, Lake Formation, Athena, Redshift).
• Proficiency in Python; React/Node.js in BFF a plus.
• Understand MLOps and self-service data science capabilities.
• Experience with Kafka or Spark Streaming and SQL/data modelling.

Responsibilities

• Lead and mentor a data engineering team to deliver scalable data solutions.
• Provide hands-on technical direction, review designs and code, uphold quality standards.
• Design solutions aligned with architecture standards and governance.
• Communicate technical approaches to stakeholders and project teams.
• Optimize Spark workloads on AWS EMR for batch and real-time processing.
• Oversee streaming API services for real-time data ingestion and delivery.
• Enable self-service data access for data science teams with governed data assets.
• Promote MLOps practices, including deployment and monitoring of models.
• Contribute to Frontend-for-Backend components using React/Node.js.
• Collaborate with stakeholders and platform teams to translate business needs into scalable solutions.
• Drive data governance across quality, lineage, integrity, and security.

Skills

Data engineering leadership

Apache Spark on AWS EMR

Streaming API design

Python

SQL

Data modeling

MLOps

Kafka

Communication & coaching

Education

Bachelor's degree in CS/Engineering/Data Science

Tools

Spark

Kafka

React

Node.js

AWS Lambda

Job description

Michael Page International (Hong Kong) Limited seeks a Data Engineer Lead to guide a data engineering team and shape scalable data platforms. You will drive technical direction, review designs, and ensure delivery excellence across Spark on AWS EMR and streaming APIs.

You will promote MLOps, collaborate with data science teams, and contribute to governance, security, and cost-efficient solutions while mentoring engineers and engaging stakeholders.