Lead Databricks Architect & Data Engineer

PERSOL Outsourcing; Singapore; On-site; SGD 210,000 - 270,000; Full time

Job summary

PERSOL Outsourcing is seeking a highly experienced Databricks Architect for a 1-year engagement in Central Singapore. You will lead technical delivery, set engineering standards, and mentor data engineers across multiple Databricks initiatives on AWS.

You will define Lakehouse architectures, build production pipelines with PySpark, Python, SQL and Delta Lake, and enable AI/ML solutions while ensuring security, governance and operational readiness.

Qualifications

• Bachelor's degree in CS/IT/SE or related field.
• 10+ years of experience in data engineering and data-platform design.
• 5+ years Databricks experience including production deployment and optimization.
• 2+ years as Databricks architect or equivalent architecture responsibilities.
• 2+ years delivering AI/ML or generative AI workloads on Databricks in production.
• 5+ years AWS experience supporting data platforms.
• Databricks certification required; Databricks Certified Data Engineer Professional preferred.
• Strong Apache Spark, Python/PySpark, SQL, Delta Lake, and lakehouse architecture knowledge.
• Experience with Databricks orchestration, Unity Catalog, RBAC, metadata, governance, performance tuning, production ops.
• Experience with AI workloads on Databricks: MLflow, Model Serving, vector search, RAG.

Responsibilities

• Lead the technical delivery of multiple Databricks initiatives, setting engineering standards, reviewing designs and code, and mentoring data engineers.
• Act as the Databricks architect, translating requirements into secure, scalable architectures on AWS.
• Define Lakehouse architecture, workspace strategy, data integration patterns, governance and security controls, compute strategy, and operations readiness.
• Design and build production data pipelines using Python, PySpark, SQL and Delta Lake across Bronze-Silver-Gold layers.
• Lead hands-on development of data models, reusable components and data-quality controls.
• Design and enable production AI/ML and generative AI solutions on Databricks, including feature and data pipelines, model integration, evaluation, deployment and monitoring.
• Integrate Databricks with AWS services and enterprise systems; work with IAM, networking, storage and deployment architecture.
• Optimize Spark workloads for performance and cost; resolve platform/pipeline issues.
• Establish Git-based CI/CD, automated testing, observability and release practices; maintain docs.
• Collaborate with stakeholders to estimate effort, manage dependencies and communicate design decisions.

Skills

Leadership & mentoring

Stakeholder communication

Problem solving

Project management

Education

Bachelor's degree in CS/IT/SE or related

Tools

Databricks

AWS

Python

PySpark

SQL

Delta Lake

Spark

Git-based CI/CD

Terraform

MLflow

Model Serving

Vector search

Job description

PERSOL Outsourcing is seeking a highly experienced Databricks Architect for a 1-year engagement in Central Singapore. You will lead technical delivery, set engineering standards, and mentor data engineers across multiple Databricks initiatives on AWS.

You will define Lakehouse architectures, build production pipelines with PySpark, Python, SQL and Delta Lake, and enable AI/ML solutions while ensuring security, governance and operational readiness.