Hybrid Data Engineer – Kuala Lumpur, AI‑Driven Pipelines

The Access Group; Kuala Lumpur; Hybrid; MYR 120,000 - 210,000; Full time

Benefits offered by this job

25 days leave

Health contribution

3 charity days allocated

Job summary

The Access Group in Kuala Lumpur is hiring a Data Engineer to design, build, and operate scalable data pipelines on Databricks, ingesting data from APIs, and modeling data for analytics. The role emphasizes AI-assisted tooling, automation, and collaboration with cross-functional teams.

You will work in a hybrid setup, contributing to data platform enhancements, governance, and reliable data delivery while aligning with business goals and stakeholder needs.

Qualifications

• 3–5 years of professional experience in Data Engineering.
• Strong hands-on experience with Databricks, Spark/PySpark, SQL, Delta Lake and Databricks Workflows.
• Experience building API-driven data ingestion pipelines.
• Solid understanding of dimensional modelling and layered/medallion data architectures.
• Proficiency in Python and SQL.
• Strong communication and stakeholder management skills.

Responsibilities

• Design, build, and operate reliable data pipelines on Databricks from ingestion through to curated, consumption-ready layers.
• Ingest data from internal and third-party APIs, managing authentication, pagination, schema evolution, rate limiting, retries, and incremental loading.
• Develop and maintain scalable data models using dimensional and medallion architecture principles to support analytics and downstream applications.
• Apply modern data engineering practices, including modular development, automated testing, orchestration, CI/CD pipelines, version control, and environment promotion.
• Monitor pipeline performance, quality, reliability, and cost efficiency, resolving issues proactively.
• Collaborate with product, engineering, analytics, and business teams to deliver robust end-to-end data solutions.

Skills

Databricks

Spark/PySpark

SQL

Delta Lake

Databricks Workflows

Python

APIs ingestion

Dimensional modelling

Medallion architecture

CI/CD

Git

Azure DevOps

Tools

Azure DevOps

Git

Neo4j

FastAPI

Job description

The Access Group in Kuala Lumpur is hiring a Data Engineer to design, build, and operate scalable data pipelines on Databricks, ingesting data from APIs, and modeling data for analytics. The role emphasizes AI-assisted tooling, automation, and collaboration with cross-functional teams.

You will work in a hybrid setup, contributing to data platform enhancements, governance, and reliable data delivery while aligning with business goals and stakeholder needs.