Senior Data Engineer - Azure Databricks & DevOps Up to 73k

Randstad Hong Kong Limited; Hong Kong; On-site; HKD 700,000 - 900,000; Full time

Job summary

Randstad Hong Kong Limited is looking for a highly skilled Lead Data Engineer to join their technology hub in Hong Kong. This role involves building robust ETL pipelines, and evaluating technologies to drive enterprise-level transformation. The ideal candidate will have significant experience in modern cloud data architectures and a strong preference for Azure Databricks.

Joining a collaborative environment, the Lead Data Engineer will work closely with Data Architects and Scientists to ensure optimal platform health and elevate data governance standards within the company.

Qualifications

• Minimum 10 years of experience in Data Engineering or related fields.
• Proven expertise in cloud data architectures, especially in Azure.
• Deep hands-on experience with cloud ecosystems and governance frameworks.

Responsibilities

• Architect and maintain high-performing ETL pipelines.
• Collaborate with teams to deliver tailored data solutions.
• Leverage AI-assisted tools to enhance development cycles.

Skills

Data Engineering

Cloud data architectures

Azure Databricks

Python

SQL

Terraform

Git

CI/CD workflows

Education

Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Science

Tools

Azure DevOps

Kubernetes

GitHub Copilot

Job description

Are you a strategic data professional ready to drive enterprise-level transformation? We are seeking a highly skilled Lead Data Engineer to partner with cross-functional teams in Hong Kong. In this role, you will evaluate emerging technologies, engineer scalable cloud services, and build robust ETL pipelines. Serving as a crucial bridge between data engineering, operations, and business strategy, the successful candidate will empower stakeholders to harness data for actionable insights and commercial success.

You will be joining a high-performing, cross-functional technology hub comprising Data Architects, Data Scientists, and Data Engineers. Reporting directly to the Head of Technical Delivery, this highly collaborative team champions knowledge sharing, secure-by-design principles, and continuous technical upskilling within a large-scale enterprise environment.

Responsibilities

• Act as the hands-on technical anchor for Data Engineering and DevOps initiatives.
• Architect, scale, and maintain high-performing ETL pipelines for both structured and unstructured data, ensuring strict data reliability.
• Design and optimise CI/CD workflows covering ETL processes, data assets, cloud infrastructure, and web applications.
• Evaluate and enhance the data platform to maximise cost-efficiency and improve the developer experience (e.g., monitoring cloud expenditure).
• Serve as a core platform administrator, collaborating with business analysts, project managers, and data scientists to deliver tailored data solutions.
• Champion best practices in cloud engineering, data security, and DevOps methodologies across the team.
• Leverage modern AI-assisted tools to accelerate development cycles and boost overall delivery productivity.
• Partner with external vendors and IT service providers for solution architecture, technical support, and BAU system delivery.
• Define, track, and visualise key performance indicators (KPIs) to ensure optimal platform health.
• Elevate enterprise data governance standards and data quality frameworks across the analytics ecosystem.

Skills & Experience Required

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Science, or a closely related technical discipline.
• Minimum 10 years of hands-on experience spanning Data Engineering, Data Analytics, Platform Engineering, or DevOps. (Candidates demonstrating exceptional potential but with fewer years of experience may be considered for a Mid-Level Data Engineer capacity).
• Proven expertise in modern cloud data architectures, with a strong preference for Azure Databricks (PySpark) and governance frameworks (e.g., Unity Catalog).
• Deep hands-on experience with cloud ecosystems, specifically Microsoft Azure.
• High proficiency in Git, Jira, Azure DevOps, Python, SQL, shell scripting, and Terraform.
• Familiarity with containerisation and orchestration technologies (e.g., Kubernetes, Helm, Istio) is a distinct advantage.
• Prior exposure to Oracle ERP deployments or modern web application frameworks (e.g., Spring Boot, FastAPI, Vue.js) is highly desirable.
• Proven experience utilising AI coding assistants (e.g., GitHub Copilot, OpenAI Codex, Claude Code) to enhance individual and team-level efficiency.
• Exceptional sense of ownership, proactive problem-solving abilities, and robust time-management skills.
• Outstanding communication and translation skills, with the ability to demystify complex technical concepts for diverse commercial stakeholders.