DATA ARCHITECT - Databricks

Happiest Minds Technologies; Kuala Lumpur; On-site; MYR 320,000 - 520,000; Full time

Job summary

TNB Tenaga Malaysia is seeking a Solution Architect to lead the end-to-end Cloudera-to-Databricks migration, aligning with RT25 and ET Aspirations. This onsite role bridges client stakeholders, AWS/Databricks architects, and offshore teams to ensure a scalable Lakehouse-based solution across discovery, design, migration, testing, and deployment.

Key focus areas include governance, security, data ingestion, and migration strategies for Hive/Impala/Spark/Oozie/Informatica, with leadership in

Qualifications

• 15+ years in Data & Analytics solutions.

Responsibilities

• Lead architecture discovery workshops with business and technical stakeholders.
• Define target-state Databricks Lakehouse architecture on AWS.
• Design data ingestion, transformation, storage, governance, security, and consumption layers.
• Establish migration strategy for Hive, Impala, Spark, Oozie, Informatica, and reporting workloads.
• Define integration architecture with source systems, APIs, databases, and streaming platforms.
• Review and approve technical designs delivered by offshore teams.
• Provide architecture governance and ensure adherence to enterprise standards.
• Support estimation, risk assessment, and technical decision-making.
• Collaborate with AWS, Databricks, and client infrastructure teams during implementation.
• Lead architecture reviews, UAT readiness reviews, cutover planning, and production go-live activities.
• Act as the primary escalation point for technical challenges and architectural risks.

Skills

Databricks Lakehouse

Apache Spark PySpark

Delta Lake

Unity Catalog

AWS Cloud Architecture

Hadoop Ecosystem

Informatica Migration

Data Warehousing

Security Architecture

Data Integration

Microservices & API

Enterprise Architecture

Stakeholder Management

Executive Communication

Presales & Solutioning

Risk & Dependency Mgmt

Team Leadership

Databricks Certification

Tools

Databricks

AWS

Job description

TNB Tenaga Malaysia to migrate its existing Cloudera data ecosystem to the modern Databricks Data Intelligence Platform (Lakehouse Architecture). This migration is not a mere technical uplift, but a critical enabler aligned directly with TNB's Reimagining TNB 2025 (RT25) strategy and its overarching Energy Transition (ET) Aspiration Plan, specifically supporting the Grid of the Future and Sustainability Pathway 2050 pillars.TNB?s existing Cloudera environment (v7.1.7) to the Databricks Lakehouse Platform using a ?Lift-and-Shift, then Modernize? approach. This approach ensures minimal operational disruption during migration while establishing a scalable foundation for advanced analytics, AI, and domain-led modernization. Solution Architect (Onsite? Malaysia)

• 1. Role Summary
• The Solution Architect serves as the primary technical authority for the engagement and acts as the bridge between TNB stakeholders, AWS/Databricks architects, and offshore delivery teams. The role is responsible for defining the end-to-end solution architecture, validating technical decisions, managing architecture governance, and ensuring successful execution of the Cloudera-to-Databricks migration program. The architect will work closely with client IT, infrastructure, security, and business teams throughout discovery, design, migration, testing, and deployment phases
• Key Responsibilities
• Lead architecture discovery workshops with business and technical stakeholders.
• Define target-state Databricks Lakehouse architecture on AWS.
• Design data ingestion, transformation, storage, governance, security, and consumption layers.
• Establish migration strategy for Hive, Impala, Spark, Oozie, Informatica, and reporting workloads.
• Define integration architecture with source systems, APIs, databases, and streaming platforms.
• Review and approve technical designs delivered by offshore teams.
• Provide architecture governance and ensure adherence to enterprise standards.
• Support estimation, risk assessment, and technical decision-making.
• Collaborate with AWS, Databricks, and client infrastructure teams during implementation.
• Lead architecture reviews, UAT readiness reviews, cutover planning, and production go-live activities.
• Act as the primary escalation point for technical challenges and architectural risks.
• 2. Required Skills
• 1. Technical Skills
• Databricks Lakehouse Platform
• Apache Spark, PySpark, Structured Streaming
• Delta Lake and Delta Live Tables
• Unity Catalog and Data Governance
• AWS Cloud Architecture
• Hadoop Ecosystem (Hive, HDFS, Impala, Oozie)
• Informatica Migration
• Data Warehousing and Modern Data Platforms
• Lakehouse Architecture Design
• Security Architecture and Data Protection
• Data Integration Patterns
• Microservices and API Integration
• 2. Leadership Skills
• Enterprise Architecture Governance
• Stakeholder Management
• Executive Communication
• Technical Solutioning and Presales
• Risk and Dependency Management
• Team Leadership and Mentoring
• 3. Preferred Experience
• 15+ years in Data & Analytics solutions.
• 5+ years architecting Databricks platforms.
• Experience delivering large-scale cloud migrations.
• Prior utility, energy, or critical infrastructure industry experience preferred.
• AWS Solution Architect Professional certification preferred.
• Databricks Certified Data Engineer/Architect preferred.

Databricks, AWS Migration, PySpark