Senior Databricks Solution Architect

Kaliba; Sydney; On-site; AUD 180,000 - 240,000; Full time

Job summary

Kaliba in Sydney seeks a Senior Databricks Solution Architect to design and deliver enterprise Databricks Lakehouse platforms, lead complex implementations and provide technical direction across major data modernization initiatives.

You will combine hands-on Databricks, Spark and cloud data platform expertise with strong solution design and stakeholder-management skills to drive scalable, secure, and cost-efficient data platforms for enterprise clients.

Qualifications

• Strong hands-on Databricks architecture experience.
• Proven delivery of enterprise-scale Databricks solutions.
• Deep knowledge of Apache Spark, PySpark, SQL and Delta Lake.
• Strong understanding of Lakehouse and Medallion architecture.
• Experience with enterprise ETL/ELT, data modelling and data engineering.
• Azure, AWS or Google Cloud data-platform experience.
• Knowledge of Databricks Workflows, Jobs, Notebooks, Repos and cluster management.
• Experience implementing CI/CD practices for modern data platforms.
• Strong troubleshooting, performance-tuning and cost-optimisation skills.
• Excellent solution-design, documentation and stakeholder-management capability.

Responsibilities

• Designing enterprise Databricks Lakehouse solutions.
• Defining workspace architecture, cluster strategy, security models and operational standards.
• Designing Delta Lake and Medallion architectures across Bronze, Silver and Gold layers.
• Architecting scalable data pipelines using Databricks, Apache Spark, PySpark and SQL.
• Leading Databricks implementations, migrations and production deployments.
• Improving platform performance, reliability and cost efficiency.
• Establishing data-quality frameworks and schema-evolution strategies.
• Implementing governance through Unity Catalog, access controls, lineage and auditing.
• Producing architecture documentation and leading technical design discussions.
• Working with engineering teams and enterprise stakeholders throughout delivery.

Skills

Databricks architecture

Enterprise Databricks deployments

Apache Spark

PySpark

SQL

Delta Lake

Lakehouse/Medallion architecture

ETL/ELT

Data modelling

Cloud platforms (Azure/AWS/GCP)

CI/CD for data platforms

Performance tuning

Stakeholder management

Job description

Senior Databricks Solution Architect | Sydney

We’re partnering with a specialist technology consultancy that is expanding its Databricks capability across Australia and the wider APAC region.

This is an opportunity to design and deliver enterprise-scale Databricks Lakehouse platforms, lead complex implementations and provide technical direction across major data‑modernisation initiatives.

We’re looking for an experienced architect who combines strong solution-design capability with genuine hands‑on expertise across Databricks, Spark and modern cloud data platforms.

What you’ll be doing

• Designing enterprise Databricks Lakehouse solutions
• Defining workspace architecture, cluster strategy, security models and operational standards
• Designing Delta Lake and Medallion architectures across Bronze, Silver and Gold layers
• Architecting scalable data pipelines using Databricks, Apache Spark, PySpark and SQL
• Leading Databricks implementations, migrations and production deployments
• Improving platform performance, reliability and cost efficiency
• Establishing data-quality frameworks and schema-evolution strategies
• Implementing governance through Unity Catalog, access controls, lineage and auditing
• Producing architecture documentation and leading technical design discussions
• Working with engineering teams and enterprise stakeholders throughout delivery

What we’re looking for

• Strong hands‑on Databricks architecture experience
• Proven delivery of enterprise‑scale Databricks solutions
• Deep knowledge of Apache Spark, PySpark, SQL and Delta Lake
• Strong understanding of Lakehouse and Medallion architecture
• Experience with enterprise ETL/ELT, data modelling and data engineering
• Azure, AWS or Google Cloud data‑platform experience
• Knowledge of Databricks Workflows, Jobs, Notebooks, Repos and cluster management
• Experience implementing CI/CD practices for modern data platformsStrong troubleshooting, performance‑tuning and cost‑optimisation skills
• Excellent solution‑design, documentation and stakeholder‑management capability

Highly regarded

• Unity Catalog and enterprise data‑governance experience
• MLflow or MLOps experience
• Experience supporting AI and machine‑learning workloads
• Data‑platform migration and Lakehouse‑transformation experience

Why consider it?

• Permanent position based in Sydney
• Australian‑standard employee benefits
• Opportunity to work on enterprise Databricks transformation programs
• Influence over platform architecture, governance and engineering standards
• Hands‑on technical work combined with strategic solution design
• Exposure to modern data, cloud and AI‑enabled platforms
• Opportunity to contribute to a growing Databricks capability

Candidates must be available immediately or have a maximum notice period of 30 days.

The selection process includes two technical interviews followed by one client interview with Databricks.