Databricks Architect & Lead Data Engineer

PERSOL TECH SERVICES PTE. LTD.; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

PERSOL TECH SERVICES PTE. LTD. is seeking a seasoned Databricks Architect to lead the technical delivery of multiple Databricks initiatives on AWS, translate requirements into secure, scalable architectures, and mentor data engineers.

The role covers designing Lakehouse architecture, building production data pipelines with Python, PySpark, SQL and Delta Lake, enabling AI/ML workloads, and ensuring governance, IAM, and cost optimization across the platform.

Qualifications

• Bachelor's degree or equivalent in CS/IT/SE or related field.
• 10+ years of data engineering experience with enterprise data-platform design.
• 5+ years hands-on Databricks experience including production implementations.
• 2+ years as a Databricks architect or equivalent.
• 2+ years delivering AI/ML or generative AI workloads on Databricks.
• 5+ years AWS experience supporting data platforms.
• Databricks certification required; additional ML/GenAI certs advantageous.
• Deep expertise in Spark, PySpark, SQL, Delta Lake and lakehouse architecture.
• Experience with Databricks orchestration, Unity Catalog, RBAC, metadata, lineage and governance.
• Hands-on with CI/CD, testing and deployment; Terraform experience advantageous.

Responsibilities

• Lead the technical delivery of multiple Databricks initiatives and mentor data engineers.
• Act as the Databricks architect translating requirements into secure, scalable architectures on AWS.
• Define Databricks Lakehouse architecture, workspace strategy and security controls.
• Design and build production data pipelines using Python, PySpark, SQL and Delta Lake.
• Lead development of data models, reusable components and data-quality controls.
• Design and enable production AI/ML and generative AI solutions on Databricks.
• Integrate Databricks with AWS services and enterprise systems, coordinating IAM and networking.
• Optimize Spark workloads for performance, reliability and cost.
• Establish Git-based CI/CD, automated testing, observability and release practices.
• Collaborate with stakeholders to estimate effort and communicate design decisions across initiatives.

Skills

Leadership

Mentoring

Architectural design

Communication

Problem-solving

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

AWS

Python

PySpark

SQL

Delta Lake

Unity Catalog

MLflow

Terraform

S3

IAM

Job description

PERSOL TECH SERVICES PTE. LTD. is seeking a seasoned Databricks Architect to lead the technical delivery of multiple Databricks initiatives on AWS, translate requirements into secure, scalable architectures, and mentor data engineers.

The role covers designing Lakehouse architecture, building production data pipelines with Python, PySpark, SQL and Delta Lake, enabling AI/ML workloads, and ensuring governance, IAM, and cost optimization across the platform.