Data Migration Solution Architect

Beyondsoft Singapore; Kuala Lumpur; On-site; MYR 180,000 - 360,000; Full time

Job summary

Beyondsoft Singapore is seeking an experienced Data/Cloud Architect to own end-to-end Databricks architecture for enterprise migration programmes. Focus areas include Cloudera/Hadoop-to-Databricks modernisation, security governance, resilience and performance.

You will lead discovery, target-state lakehouse design, and migration waves, coordinating across security, governance and delivery teams to obtain approvals.

Qualifications

• Hands-on architecture experience with Databricks and lakehouse patterns.
• Strong knowledge of Cloudera/Hadoop ecosystems (HDFS, Hive, Impala, Spark).
• Understanding of data security, IAM, encryption, governance, lineage and regulatory requirements.
• Excellent stakeholder management and documentation skills.
• Banking or financial-services delivery experience preferred.

Responsibilities

• Lead discovery and current-state assessment of data platforms, workloads, integrations, security controls and dependencies.
• Define target-state Databricks Lakehouse architecture and migration waves.
• Design integration patterns across batch, streaming, APIs and cloud services.
• Develop migration principles covering data movement, ETL/ELT and cutover.
• Perform sizing, capacity, performance, resilience and cost considerations.
• Produce architecture diagrams, design documents and roadmaps.
• Collaborate with client teams to obtain design alignment and approvals.

Skills

Databricks architecture

Spark/PySpark

Delta Lake

Lakehouse patterns

Security governance

Tools

HDFS

Hive

Impala

Spark

Kafka

NiFi

Azure Databricks

ADF

ADLS

Job description

Primary Scope: Databricks architecture, migration assessment and design, lakehouse, integration, security, performance, and stakeholder solutioning.

Own the end-to-end architecture and solution design for enterprise data-platform migration programmes, with a strong focus on Cloudera/Hadoop-to-Databricks modernisation and banking-grade security, governance, resilience and performance.

Key Responsibilities

• Lead discovery and current-state assessment of data platforms, workloads, integrations, security controls and operational dependencies.
• Define target-state Databricks Lakehouse architecture, migration waves, workload disposition and coexistence approach.
• Design integration patterns across batch, streaming, APIs, cloud services, identity, security, monitoring and downstream consumers.
• Develop migration principles covering data movement, ETL/ELT conversion, validation, reconciliation, cutover, rollback and post-migration stabilisation.
• Perform sizing, capacity, performance, resilience and cost considerations for the target platform.
• Produce architecture diagrams, solution design documents, capability mappings, migration roadmaps and technical decision records.
• Work with client architecture, security, data governance, infrastructure, application and delivery teams to obtain design alignment and approvals.

Required Skills & Experience

• Hands-on architecture experience with Databricks, Spark/PySpark, Delta Lake and modern lakehouse patterns.
• Strong knowledge of Cloudera/Hadoop ecosystems such as HDFS, Hive, Impala, Spark, Kafka and NiFi.
• Good understanding of data security, IAM, encryption, network controls, governance, lineage, auditability and regulatory requirements.
• Strong stakeholder management, workshop facilitation, documentation and presentation skills.
• Banking or financial-services delivery experience is preferred.

Preferred / Advantageous

• Databricks certifications or equivalent cloud/data architecture certifications.
• Experience with Azure Databricks, ADF, ADLS and enterprise CI/CD.