Lead Data Engineer

CLP; Hong Kong; On-site; HKD 600,000 - 750,000; Full time

Job summary

CLP in Hong Kong is seeking an experienced Data Engineer to enhance and maintain a robust data platform. The role involves collaborating closely with cross-functional teams to optimize data pipelines and implement cloud solutions.

Candidates should possess a Bachelor’s or Master’s degree and at least 10 years of relevant experience in data engineering and analytics. The position demands expertise in Azure cloud services, along with strong problem-solving and communication skills.

Qualifications

• 10+ years of hands-on experience in data analytics, data engineering, platform engineering, or DevOps.
• Proven expertise in cloud data analytics, especially Azure Databricks with pyspark.
• Strong communication skills to engage with stakeholders effectively.

Responsibilities

• Build, maintain, and optimise ETL pipelines for structured and unstructured data.
• Collaborate with teams to design and deliver data solutions aligned with business goals.
• Track and visualise KPIs to monitor data platform performance.

Skills

Data Engineering

DevOps

Python programming

SQL

Cloud services (Azure)

Data governance

Teamwork

Education

Bachelor’s or master’s degree in a related field

Tools

Azure Databricks

Jira

Git

Azure DevOps

Job description

Employment duration: Permanent

Location: Kai Tak

You will work closely with Data Architects, Data Scientists, and Data Engineers to enhance and maintain a data platform supporting diverse business models within CLP.

This includes conducting feasibility assessments to identify appropriate tools and technologies, developing and optimizing shared platform services, and building data pipelines to ingest, integrate, and manage enterprise data for downstream analytical use by different technical and business stakeholders to generate actionable insights.

Responsibilities

• Hands‑on experience in Data Engineering and DevOps (required).
• Build, maintain, and optimise data ETL pipelines for structured and unstructured data while ensuring scalability, data quality, and reliability.
• Build, maintain, and optimise CI/CD pipelines for ETL workflows, data assets, cloud infrastructure, and web applications to integrate the data asset landscape with new ERP systems and drive continued adoption of modern AI technologies.
• Identify and implement enhancements for the data platform to improve cost efficiency and developer experience (e.g., conduct analyses of existing cloud usage and coordinate within the team to manage cloud costs).
• As a data platform administrator, collaborate with data scientists, data engineers, business analysts, project managers, and various business stakeholders to design and deliver data solutions aligned with the latest business goals.
• Share cloud engineering, DevOps, and data platform expertise with teammates who specialise in other areas to ensure successful project delivery and adherence to best practices and compliance requirements (e.g., secure by design).
• Use AI tools to boost development and delivery productivity.
• Collaborate with vendors and service providers on solution planning, service delivery, and technical support activities; provide BAU support for IT systems.
• Track and visualise KPIs to monitor data platform performance.
• Perform other ad hoc tasks as assigned by the supervisor.
• Uplift data governance practices and data quality management in the use of data analytics platform.

Requirements

Academic Qualification

• Bachelor’s or master’s degree in a related field (e.g., computer science, information technology, data science, etc.).

Professional Experience

• Bachelor’s degree in information technology, computer science, or a related discipline.
• At least 10 years of hands‑on experience in data analytics, data engineering, platform engineering, DevOps, or related fields. If the candidate has less than 10 years’ experience we can consider him/her for a data engineer position.
• Proven expertise in cloud data analytics, preferably Azure Databricks with pyspark, including platform operations, data governance via Unity Catalog, and integrations and connections between different cloud and other services.
• Hands‑on experience with cloud services (preferably Azure).
• Proficiency in Jira, Git, and Azure DevOps. Experience with containerisation is an advantage (e.g., Kubernetes, Helm, Istio).
• Hands‑on experience with Python programming, SQL, shell scripting, and terraform.
• Good-to‑have hands‑on experience with Oracle ERP solutions, web application development (e.g., Vue, Spring Boot, FastAPI).
• Experience on using AI tools to uplift development productivity of individuals and team (e.g., Github Copilot, Claude Code, OpenCode, Codex).
• A strong sense of proactive teamwork, ownership, problem‑solving, and time management.
• Excellent communication skills to explain concepts to stakeholders of all backgrounds and levels of proficiency.