DATA ARCHITECT - Databricks

Happiest Minds Technologies; Kuala Lumpur; On-site; MYR 300,000 - 540,000; Full time

Job summary

TNB Tenaga Malaysia is seeking a Solution Architect to lead the end-to-end migration from Cloudera to the Databricks Lakehouse Platform on AWS. The role acts as the technical authority, coordinating between stakeholders, cloud architects, and offshore delivery teams through discovery, design, migration, testing, and deployment phases.

The ideal candidate has 15+ years in Data & Analytics solutions and 5+ years architecting Databricks platforms, with experience in large cloud migrations and

Qualifications

• 15+ years in Data & Analytics solutions.
• 5+ years architecting Databricks platforms.
• Experience delivering large-scale cloud migrations.
• AWS Solution Architect Professional certification preferred.
• Databricks Certified Data Engineer/Architect preferred.

Responsibilities

• Lead architecture discovery workshops with business and technical stakeholders.
• Define target-state Databricks Lakehouse architecture on AWS.
• Design data ingestion, transformation, storage, governance, security, and consumption layers.
• Establish migration strategy for Hive, Impala, Spark, Oozie, Informatica, and reporting workloads.
• Define integration architecture with source systems, APIs, databases, and streaming platforms.
• Review and approve technical designs delivered by offshore teams.
• Provide architecture governance and ensure adherence to enterprise standards.
• Support estimation, risk assessment, and technical decision-making.
• Collaborate with AWS, Databricks, and client infrastructure teams during implementation.
• Lead architecture reviews, UAT readiness reviews, cutover planning, and production go-live activities.
• Act as the primary escalation point for technical challenges and architectural risks.

Skills

Databricks Lakehouse Platform

Apache Spark

PySpark

Structured Streaming

Delta Lake

Unity Catalog

AWS Cloud Architecture

Hadoop Ecosystem

Informatica Migration

Data Warehousing

Lakehouse Architecture

Security Architecture

Data Integration Patterns

API Integration

Microservices

Tools

Databricks Lakehouse Platform

Apache Spark

PySpark

Structured Streaming

Delta Lake

Unity Catalog

AWS

Job description

TNB Tenaga Malaysia

to migrate its existing Cloudera data ecosystem to the modern Databricks Data Intelligence Platform (Lakehouse Architecture). This migration is not a mere technical uplift, but a critical enabler aligned directly with TNB's Reimagining TNB 2025 (RT25) strategy and its overarching Energy Transition (ET) Aspiration Plan, specifically supporting the Grid of the Future and Sustainability Pathway 2050 pillars.TNB?s existing Cloudera environment (v7.1.7) to the Databricks Lakehouse Platform using a ?Lift-and-Shift, then Modernize? approach. This approach ensures minimal operational disruption during migration while establishing a scalable foundation for advanced analytics, AI, and domain-led modernization.

Solution Architect (Onsite? Malaysia)

Role Summary

The Solution Architect serves as the primary technical authority for the engagement and acts as the bridge between TNB stakeholders, AWS/Databricks architects, and offshore delivery teams. The role is responsible for defining the end-to-end solution architecture, validating technical decisions, managing architecture governance, and ensuring successful execution of the Cloudera-to-Databricks migration program. The architect will work closely with client IT, infrastructure, security, and business teams throughout discovery, design, migration, testing, and deployment phases.

Key Responsibilities

• Lead architecture discovery workshops with business and technical stakeholders.
• Define target-state Databricks Lakehouse architecture on AWS.
• Design data ingestion, transformation, storage, governance, security, and consumption layers.
• Establish migration strategy for Hive, Impala, Spark, Oozie, Informatica, and reporting workloads.
• Define integration architecture with source systems, APIs, databases, and streaming platforms.
• Review and approve technical designs delivered by offshore teams.
• Provide architecture governance and ensure adherence to enterprise standards.
• Support estimation, risk assessment, and technical decision-making.
• Collaborate with AWS, Databricks, and client infrastructure teams during implementation.
• Lead architecture reviews, UAT readiness reviews, cutover planning, and production go-live activities.
• Act as the primary escalation point for technical challenges and architectural risks.

Required Skills

Technical Skills

• Databricks Lakehouse Platform
• Apache Spark, PySpark, Structured Streaming
• Delta Lake and Delta Live Tables
• Unity Catalog and Data Governance
• AWS Cloud Architecture
• Hadoop Ecosystem (Hive, HDFS, Impala, Oozie)
• Informatica Migration
• Data Warehousing and Modern Data Platforms
• Lakehouse Architecture Design
• Security Architecture and Data Protection
• Data Integration Patterns
• Microservices and API Integration

Leadership Skills

• Enterprise Architecture Governance
• Stakeholder Management
• Executive Communication
• Technical Solutioning and Presales
• Risk and Dependency Management
• Team Leadership and Mentoring

Preferred Experience

• 15+ years in Data & Analytics solutions.
• 5+ years architecting Databricks platforms.
• Experience delivering large-scale cloud migrations.
• Prior utility, energy, or critical infrastructure industry experience preferred.
• AWS Solution Architect Professional certification preferred.
• Databricks Certified Data Engineer/Architect preferred.

Tags

Databricks, AWS Migration, PySpark