Senior Data engineer (Azure & Databricks)

Randstad Hong Kong Limited; Hong Kong; On-site; HKD 500,000 - 700,000; Full time

Job summary

Randstad Hong Kong Limited is looking for an experienced Data Engineer based in Hong Kong. The successful candidate will build end-to-end data pipelines, collaborate with business teams, and design analytics solutions to meet technical performance metrics.

Applicants should possess a Bachelor’s or Master’s degree in a relevant field, along with a minimum of 5 years of experience in SQL, data engineering, and Business Intelligence solutions. Proficiency in Databricks and Azure cloud platform is mandatory.

A strong understanding of programming languages such as Python and advanced SQL scripting is also essential.

Qualifications

• Minimum of 5 years in SQL, data engineering, and Business Intelligence (BI) solutions.
• High proficiency in Python, Spark, Scala, and advanced SQL scripting.
• Strong knowledge of Data Lakes, Data Warehouse, and BI Dashboards.

Responsibilities

• Build end-to-end data pipelines including data collection, transformation, quality, and integration.
• Collaborate with business teams to identify data requirements and assemble complex datasets.
• Design and implement analytics solutions to meet technical performance metrics.
• Maintain data model integrity and data governance.
• Support stable operations with consistent pipeline deployments.

Skills

SQL

Python

Data Engineering

Business Intelligence

Azure

Education

Bachelor’s or Master’s degree in Computer Science, IT, or a related field

Tools

Databricks

PostgreSQL

HBase

MongoDB

Cassandra

Job description

About the Job

Key Responsibilities

• Pipeline Engineering: Build end-to-end data pipelines including data collection, transformation, quality, and integration.
• Solution Design: Collaborate with business teams to identify data requirements and assemble large, complex datasets.
• Optimization: Design, implement, and fine-tune analytics solutions to meet technical performance metrics.
• Integrity & Security: Work with the Data Architect to maintain data model integrity and data governance.
• Operations: Partner with DevOps engineers to support consistent pipeline deployments and stable operations.
• AI & API Collaboration: Develop robust APIs and data tools to assist data scientists and machine learning engineers.

Education & Experience

• Degree: Bachelor’s or Master’s degree in Computer Science, IT, or a related field.
• Experience: Minimum of 5 years in SQL, data engineering, and Business Intelligence (BI) solutions.

Technical Stack (Mandatory)

• Databricks: Hands‑on experience delivering data projects using Databricks (Spark, notebooks, pipelines) is mandatory.
• Cloud Platform: Proven experience designing and building solutions on the Azure cloud platform.

Languages & Databases

• Languages: High proficiency in Python, Spark, Scala, and advanced SQL scripting.
• Databases: Experience with relational databases (e.g., PostgreSQL) and NoSQL platforms (e.g., HBase, MongoDB, Cassandra).
• Architecture: Strong knowledge of Data Lakes, Data Factory, Data Warehousing, and BI Dashboards.

Modern AI Tools & Concepts

• AI Productivity: Experience using coding assistants (e.g., GitHub Copilot) to uplift development speed.
• GenAI Awareness: Familiarity with GenAI concepts (RAG, LangChain, LlamaIndex, vector databases) to collaborate effectively on AI use cases.