Chief Architect - Data Engineering (Databricks Practice)

Kaliba; Council of the City of Sydney; On-site; AUD 180,000 - 260,000; Full time

Job summary

Kaliba is partnering with a specialist technology consultancy building a high-growth Databricks practice across Australia and the APAC region. This is your chance to become the technical anchor of the practice—owning enterprise architecture, leading major transformation programs and helping build the capability from the ground up.

This role requires hands-on leadership, presenting to CTOs and architecture boards while diagnosing Spark performance issues and guiding engineering teams through

Qualifications

• 12+ years in data engineering or data architecture.
• 3+ years delivering Databricks at enterprise scale.
• Deep Spark, Delta Lake and Lakehouse architecture expertise.
• Strong Python and SQL skills; Scala is a plus.
• Experience with Unity Catalog, Databricks Workflows, Airflow or ADF.
• Proven leadership on major migration or modernization programs.
• Presales, estimation and executive stakeholder capability.
• Mentoring engineers and setting architecture standards.
• Databricks certification or Solution Architect accreditation preferred.

Responsibilities

• Own end-to-end architecture across enterprise Databricks engagements.
• Design secure, scalable, high-performing Lakehouse platforms.
• Lead large-scale cloud, data warehouse, ETL and Hadoop migrations.
• Drive discovery, solution design, estimation, PoCs, RFPs and proposals.
• Present and defend architecture decisions to senior stakeholders.
• Establish engineering standards and lead design and code reviews.
• Mentor engineers and develop the Databricks capability.
• Create reusable accelerators and go-to-market offerings.
• Remain hands-on with Spark performance, debugging and optimisation.

Skills

Databricks

Apache Spark

Python

SQL

Scala

Presales & estimation

Stakeholder communication

Mentoring engineers

Tools

Unity Catalog

Databricks Workflows

Lakeflow / DLT

Airflow

Azure Data Factory (ADF)

Job description

We’re partnering with a specialist technology consultancy that is building a high-growth Databricks practice across Australia and the wider APAC region.

This is an opportunity to become the technical anchor of the practice—owning enterprise architecture, leading major transformation programs and helping build the capability from the ground up.

This is not a diagram-only architecture role. We’re looking for someone who can confidently present to CTOs and architecture boards while remaining hands‑on enough to diagnose Spark performance issues and guide engineering teams through complex delivery challenges.

What you’ll be doing

• Owning end-to-end architecture across enterprise Databricks engagements
• Designing secure, scalable and high-performing Lakehouse platforms
• Leading large-scale cloud, data warehouse, ETL and Hadoop migrations
• Driving discovery, solution design, estimation, PoCs, RFPs and proposals
• Presenting and defending architecture decisions to senior stakeholders
• Establishing engineering standards and leading design and code reviews
• Mentoring engineers and developing the wider Databricks capability
• Creating reusable accelerators and go-to-market offerings
• Remaining hands‑on with Spark performance, debugging and optimisation

What we’re looking for

• 12+ years of experience in data engineering or data architecture
• 3+ years delivering Databricks solutions at enterprise scale
• Deep expertise in Apache Spark, Delta Lake and Lakehouse architecture
• Strong Python and SQL skills, with Scala highly regarded
• Experience with Unity Catalog, Databricks Workflows, Lakeflow/DLT, Airflow or ADF
• Demonstrated leadership of major migration or modernisation programs
• Strong presales, estimation and executive stakeholder capability
• Experience mentoring engineers and establishing architecture standards
• Databricks professional certification or Solution Architect accreditation preferred

Why consider it?

• Permanent position based in Sydney
• Australian-standard employee benefits
• Opportunity to shape and grow a Databricks practice
• Influence across architecture, delivery and go-to-market strategy
• Exposure to major enterprise transformation programs
• Hands‑on technical leadership with genuine decision‑making authority

Candidates must be available immediately or have a maximum notice period of 30 days.

The selection process includes two technical interviews followed by one client interview with Databricks.