Lead Databricks Architect for Enterprise Data & AI

Unison Group; Johor Bahru; On-site; MYR 180,000 - 360,000; Full time

Job summary

Unison Group is building a fast-growing Databricks practice and seeks a hands-on Lead Solution Architect to drive enterprise-scale data, analytics, and AI solutions across APAC. You will lead architecture, discuss with CTOs, and dive into Spark tuning when needed.

Healthcare-grade lakehouse platforms, migrations from on-prem and Hadoop to Databricks on Azure/AWS/GCP, and a focus on governance, performance, and scalable delivery are core. Mandarin and/or Bahasa Malaysia proficiency is required.

Qualifications

• 12+ years of experience in Data Engineering, Data Architecture, or Cloud Data Platforms.
• 3+ years of enterprise-scale Databricks experience.
• Strong expertise in Apache Spark, performance tuning, streaming, and troubleshooting.
• Deep knowledge of Databricks Lakehouse, Delta Lake, Unity Catalog, Medallion Architecture, and governance frameworks.
• Experience with Databricks Workflows, Delta Live Tables (DLT), Lakeflow, Airflow, or Azure Data Factory.
• Proven track record leading large-scale cloud migration and modernization programs.
• Strong hands-on skills in Python, SQL, and Scala.
• Experience with CI/CD and DevOps for Data Platforms.
• Familiarity with MLflow, AI/GenAI integrations, and LLM APIs (OpenAI, Anthropic).
• Strong stakeholder management and presales experience.
• Databricks Professional Certifications highly preferred.

Responsibilities

• Lead end-to-end architecture and solution design for Databricks implementations.
• Build scalable, secure, and high-performance Lakehouse platforms.
• Deliver enterprise-grade data solutions covering ingestion, transformation, governance, orchestration, and analytics.
• Drive large-scale cloud migrations from legacy Data Warehouses, Hadoop, and on-prem platforms to Databricks on Azure, AWS, or GCP.
• Provide hands-on leadership through code reviews, performance tuning, debugging, and best-practice implementation.
• Establish engineering standards and reusable frameworks for the practice.
• Lead discovery workshops, solution assessments, PoCs, RFP responses, and proposal development.
• Present and defend solution architectures with CIOs, CTOs, and Architecture Review Boards.
• Translate complex technical concepts into clear business outcomes for executive stakeholders.
• Support effort estimation, pricing strategies, and project planning for successful deal conversion.
• Mentor and lead Data Engineers and Architects.
• Create accelerators, reusable assets, and frameworks to improve delivery efficiency.
• Convert successful client engagements into case studies and go-to-market offerings.
• Contribute to the growth and strategic direction of the Databricks practice.

Skills

Databricks

Spark

Python

SQL

Scala

Airflow

CI/CD

DevOps

Delta Lake

Delta Live Tables

Azure/AWS/GCP

MLflow

Tools

Databricks

Airflow

Delta Lake

Delta Live Tables

Azure Data Factory

Job description

Unison Group is building a fast-growing Databricks practice and seeks a hands-on Lead Solution Architect to drive enterprise-scale data, analytics, and AI solutions across APAC. You will lead architecture, discuss with CTOs, and dive into Spark tuning when needed.

Healthcare-grade lakehouse platforms, migrations from on-prem and Hadoop to Databricks on Azure/AWS/GCP, and a focus on governance, performance, and scalable delivery are core. Mandarin and/or Bahasa Malaysia proficiency is required.