Databricks Data Engineer

Peoplebank Hong Kong Limited; Hong Kong Island; On-site; HKD 600,000 - 900,000; Full time

Job summary

Peoplebank Hong Kong Limited is seeking a cloud data platform engineer to lead the implementation and optimization of scalable Azure-based data platforms. You will automate ETL/ELT workflows, enforce data quality, and collaborate with analysts to deliver near real-time analytics and BI solutions using Power BI.

Responsibilities include end-to-end project lifecycle involvement with external vendors, designing data models, validating pipelines, and producing playbooks and architecture docs for

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
• At least 3 years’ professional experience focusing on data platform implementations or cloud data migrations.
• Technical proficiency in Python, SQL, Spark, Azure Data Factory, Databricks, Unity Catalog, and Power BI.
• Hands-on experience with Event Hub, Kafka, or event messaging systems, alongside Azure Data Lake Storage Gen2.
• Clear understanding of the software development lifecycle (SDLC), data governance principles, data quality, and security.
• Professional certification in a Cloud Data platform or Databricks Certified Data Engineer credential.

Responsibilities

• Optimize, operate at scale, and enhance Microsoft Azure and Databricks cloud-based data platforms.
• Participate in the full project lifecycle, including design, development, testing, deployment, and documentation alongside external vendors.
• Collaborate with Data Analysts to prepare, cleanse, and build optimized data models and Power BI visualizations.
• Perform proofs of concept for emerging cloud data products.
• Validate data workflows, integration points, pipeline schemas, and performance benchmarks through rigorous testing.
• Create operational playbooks, automation scripts, and technical architecture documentation to streamline deployment and long-term maintainability.

Skills

Python

SQL

Spark

Azure Data Factory

Databricks

Unity Catalog

Power BI

Event Hub

Kafka

data governance

data quality

security

Education

Bachelor’s degree in Computer Science, Information Technology, Data Science, or related field

Tools

Azure Data Lake Storage Gen2

Azure Data Factory

Databricks

Unity Catalog

Power BI

Kafka

Event Hub

Job description

You will drive the implementation and optimization of a scalable Azure cloud-based data platform, enhancing enterprise data ingestion, transformation, and storage. By automating ETL/ELT workflows and enforcing data quality checks, your efforts will directly improve data reliability and reduce manual intervention. Your expertise in near real-time analytics will empower strategic decision-making and business intelligence across the enterprise.

Responsibilities

• Optimize, operate at scale, and enhance Microsoft Azure and Databricks cloud-based data platforms.
• Participate in the full project lifecycle, including design, development, testing, deployment, and documentation alongside external vendors.
• Collaborate with Data Analysts to prepare, cleanse, and build optimized data models and Power BI visualizations.
• Perform proofs of concept for emerging cloud data products.
• Validate data workflows, integration points, pipeline schemas, and performance benchmarks through rigorous testing.
• Create operational playbooks, automation scripts, and technical architecture documentation to streamline deployment and long-term maintainability.

Requirements

• Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
• At least 3 years’ professional experience focusing on data platform implementations or cloud data migrations.
• Technical proficiency in Python, SQL, Spark, Azure Data Factory, Databricks, Unity Catalog, and Power BI.
• Hands-on experience with Event Hub, Kafka, or event messaging systems, alongside Azure Data Lake Storage Gen2.
• Clear understanding of the software development lifecycle (SDLC), data governance principles, data quality, and security.
• Professional certification in a Cloud Data platform or Databricks Certified Data Engineer credential.

Kowloon City, Kowloon City District, HK