Databricks Architect & AI/ML Platform Lead

PERSOL TECH SERVICES PTE. LTD.; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

PERSOL TECH SERVICES PTE. LTD. in Singapore seeks an experienced Databricks Architect to lead the technical delivery of multiple initiatives on AWS, defining lakehouse architecture and governance.

You will design production data pipelines with PySpark, Python and Delta Lake, mentoring engineers and collaborating with security/platform teams to ensure scalable, secure solutions across Bronze to Gold layers. This is a 1-year contract based in Central Singapore.

Qualifications

• Bachelor's degree in Computer Science, IT, Software Engineering, or related field.
• 10+ years in data engineering and enterprise data-platform design and delivery.
• 5+ years hands-on Databricks experience, including production implementation, optimization and troubleshooting.
• 2+ years as a Databricks architect or performing equivalent architecture responsibilities.
• 2+ years delivering AI/ML workloads on Databricks with production experience preferred.
• 5+ years AWS experience supporting data platforms and enterprise solutions.
• Databricks certification is required; Data Engineer Professional preferred; ML/Gen AI certs advantageous.
• Deep expertise in Spark, Python/PySpark, SQL, Delta Lake and lakehouse architecture.
• Strong experience with Databricks orchestration, Unity Catalog, IAM, and governance.
• Experience with MLflow, model serving, vector search, and RAG as relevant.
• Solid AWS expertise across S3, IAM, networking, security, compute and data integration.
• Git-based CI/CD and Databricks deployment automation; Terraform advantageous.

Responsibilities

• Lead the technical delivery of multiple Databricks initiatives, setting engineering standards and mentoring data engineers.
• Act as the Databricks architect, translating business and technical requirements into secure, scalable architectures on AWS.
• Define Databricks Lakehouse architecture, workspace and environment strategy, data integration patterns, governance and security controls.
• Design and build production data pipelines using Python, PySpark, SQL and Delta Lake, covering batch, streaming and near-real-time processing.
• Lead hands-on development of data models, reusable engineering components and data-quality controls.
• Design and enable production AI/ML and generative AI solutions on Databricks with AI/ML specialists.
• Integrate Databricks with AWS services and enterprise systems, and work on IAM, networking, storage and deployment architecture.
• Optimize Spark workloads, compute and storage for performance, reliability and cost.
• Establish Git-based CI/CD, automated testing, observability and release practices for Databricks workloads.
• Work with stakeholders to estimate effort and communicate design decisions across initiatives.

Skills

Databricks architecture

AWS

Python/PySpark

SQL

Delta Lake

Data modeling

CI/CD

Mentoring

Governance & security

Spark performance tuning

Education

Bachelor's degree in Computer Science, Information Technology or related

Databricks certification (Data Engineer)

AWS certification

Tools

Databricks

Unity Catalog

Terraform

Git-based CI/CD

S3

Job description

PERSOL TECH SERVICES PTE. LTD. in Singapore seeks an experienced Databricks Architect to lead the technical delivery of multiple initiatives on AWS, defining lakehouse architecture and governance.

You will design production data pipelines with PySpark, Python and Delta Lake, mentoring engineers and collaborating with security/platform teams to ensure scalable, secure solutions across Bronze to Gold layers. This is a 1-year contract based in Central Singapore.