Data Engineering Lead / Senior Data Engineer (AWS EMR)

PERSOL; Hong Kong; On-site; HKD 900,000 - 1,300,000; Full time

Job summary

PERSOL is seeking a Senior Data Engineering Lead in Hong Kong to lead a team of data engineers, shaping architecture, governance, and delivery of critical data pipelines. You will mentor engineers, set standards, and review designs while delivering scalable, secure data solutions across batch and streaming workloads.

Strong skills in Spark on AWS EMR, streaming APIs, and a data-centric mindset are essential.

Qualifications

• Bachelor’s degree in a technical field or equivalent experience.
• 6+ years of data engineering and team leadership experience.
• Proven ability to mentor engineers, set standards and review designs and code.
• Hands-on Spark on AWS EMR with batch/streaming pipelines.

Responsibilities

• Lead and coach a data engineering team to deliver robust data pipelines.
• Set engineering standards, review designs and code for quality.
• Translate client requirements into secure, cost-efficient data solutions.
• Guide streaming API design and real-time data ingestion.
• Enable data scientists with governed, self-serve data products.
• Promote MLOps and model deployment lifecycle on the platform.
• Collaborate with stakeholders to deliver secure frontend/back-end components.

Skills

Team leadership

Data engineering

Python

Spark on EMR

React/Node.js

SQL/Data modelling

Education

Bachelor's degree in Computer Science/Engineering/Data Science

Tools

AWS EMR

S3

Glue

Lake Formation

Athena

Redshift

Step Functions

Lambda

Kafka

React

Node.js

Job description

Lead, coach and mentor a team of data engineers, growing their technical skills and delivery capability.

Provide technical leadership and hands-on guidance, setting engineering standards, reviewing designs and code, and driving quality across the team.

Contribute solution ideas and detailed design that align with the client's required architecture practice, patterns and governance.

Present and articulate technical solutions to client stakeholders and internal teams, clearly explaining design decisions and trade-offs.

Guide the team in building and optimising Spark on EMR pipelines for batch and streaming data processing.

Design and guide the handling of streaming API endpoints - real-time data ingestion, consumption and serving through APIs.

Enable data scientists to self-serve through curated data products and governed access, advising on the best delivery approach.

Champion good engineering practice around MLOps - supporting model deployment, monitoring and lifecycle management for ML models on the platform.

Support delivery of lightweight front-end and service components using React and Node.js in a BFF (Backend-for-Frontend) structure.

Partner with client stakeholders, data scientists and platform teams to translate requirements into robust, secure and cost-effective solutions.

Promote good practice in data quality, integrity, lineage and security across the data lifecycle.

Opportunity to gain exposure to AI / GenAI initiatives and contribute ideas as they emerge.

Requirements

Bachelor's degree in Computer Science, Engineering, Data Science or a related technical field, or equivalent industrial experience.

6+ years of data engineering experience, with a proven track record leading and coaching engineering teams.

Strong technical leadership skills - able to mentor engineers, set standards, and review designs and code.

Ability to present and articulate technical solutions clearly to both technical and non-technical stakeholders.

Hands-on expertise in Apache Spark on AWS EMR (key skill) - pipeline design, performance tuning and cost optimisation.

Experience designing and handling streaming API endpoints - real-time data ingestion, consumption and API-based serving.

Solid AWS data platform experience: S3, Glue, Lake Formation, Athena, Redshift, Step Functions and Lambda.

Proficiency in Python; a strong secondary skill set beyond Python is preferred - React / Node.js in a BFF structure.

Ability to contribute solution ideas and design within a client's required architecture practice, patterns and governance.

Understanding of MLOps concepts and experience enabling data scientist self-service.

Streaming and messaging experience (e.g. Kafka, Spark Streaming) and strong SQL / data modelling skills.

Interest in or exposure to AI / GenAI initiatives is a plus (not the primary focus of the role).

Excellent communication, stakeholder management and coaching skills.

Fluent in Cantonese and English; Mandarin is a plus.

Previous IT experience in a consulting firm or services vendor is preferred.