Data Solution Architect - Databricks

Showtime Consulting; City of Melbourne; On-site; AUD 130,000 - 180,000; Full time

Job summary

Showtime Consulting is seeking an experienced Senior Data Engineer / Data Architect to join a high-performing data and cloud engineering team in Australia. You will design and deliver enterprise-scale Data Lakehouse solutions on Azure Databricks, modernise legacy data platforms, and enable cloud-first analytics for reporting, analytics, ML, and operations.

The role requires deep expertise in Azure Databricks, PySpark, Delta Lake, ADLS, Synapse, and Data Governance, with a track record of

Qualifications

• Experience as a Data Engineer, Data Architect, or similar role.
• Hands-on experience with Azure Databricks.
• Proficiency with PySpark, Spark SQL, and Delta Lake.
• Experience with Azure Data Factory, ADLS, and Synapse.
• Strong data modelling experience including Dimensional Modelling and Data Vault.
• Deep understanding of data governance, security, and quality.
• Experience delivering real-time and streaming data solutions.
• Strong Azure cloud platform experience.
• Experience with Git, CI/CD, and DevOps.
• Excellent stakeholder engagement and communication skills.

Responsibilities

• Design and implement enterprise Data Lakehouse solutions using Azure Databricks.
• Lead data modernisation and cloud migration initiatives.
• Build scalable batch and real-time data processing solutions.
• Develop and maintain data pipelines using PySpark and Spark SQL.
• Define data governance, security, lineage, and data quality standards.
• Design dimensional and Data Vault data models.
• Collaborate with business and technology stakeholders to deliver strategic data platforms.
• Drive data architecture standards, best practices, and technical governance.
• Implement CI/CD and DevOps practices across data engineering workloads.
• Provide technical leadership and mentor engineering teams.

Skills

Azure Databricks

PySpark

Delta Lake

Azure Data Factory

Azure Synapse Analytics

Data Modeling

Data Governance

Dimensional Modelling

Data Vault

Cloud Architecture

CI/CD

Git

Stakeholder Engagement

Mentoring

Team Leadership

Tools

Azure DevOps

Databricks CLI

Job description

Company Description

Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, data, DevSecOps, platform engineering, AI, and digital transformation programs.

We partner with major organisations to design, modernise, and scale data platforms that enable advanced analytics, operational intelligence, regulatory compliance, and data-driven decision making.

Our teams work at the forefront of cloud data engineering and modern analytics, helping customers transform legacy data estates into scalable, cloud-native platforms built for the future.

The Role

We are seeking an experienced Senior Data Engineer / Data Architect to join a high-performing data and cloud engineering team.

This is a hands-on role focused on designing and delivering enterprise-scale Data Lakehouse solutions using Azure Databricks, modernising legacy data platforms, and enabling cloud-first analytics capabilities. You will play a key role in defining data architecture, implementing modern data engineering practices, and delivering strategic data platforms that support business-critical reporting, analytics, machine learning, and operational workloads.

The ideal candidate will bring deep expertise across Azure Databricks, PySpark, Delta Lake, Azure Data Services, Data Architecture, and Data Governance, along with a proven track record delivering large-scale data transformation and cloud migration initiatives.

Key Responsibilities

• Design and implement enterprise Data Lakehouse solutions using Azure Databricks.
• Lead data modernisation and cloud migration initiatives.
• Build scalable batch and real-time data processing solutions.
• Develop and maintain data pipelines using PySpark and Spark SQL.
• Define data governance, security, lineage, and data quality standards.
• Design dimensional and Data Vault data models.
• Collaborate with business and technology stakeholders to deliver strategic data platforms.
• Drive data architecture standards, best practices, and technical governance.
• Implement CI/CD and DevOps practices across data engineering workloads.
• Provide technical leadership and mentor engineering teams.

What You'll Need

• Experience as a Data Engineer, Data Architect, or similar role.
• Strong hands‑on experience with Azure Databricks.
• Expertise with PySpark, Spark SQL, and Delta Lake.
• Experience with Azure Data Factory (ADF), ADLS, and Azure Synapse Analytics.
• Strong data modelling experience, including Dimensional Modelling and Data Vault.
• Deep understanding of data governance, data architecture, security, and data quality.
• Experience delivering real-time and streaming data solutions.
• Strong Azure cloud platform experience.
• Experience with Git, CI/CD, and DevOps practices.
• Excellent stakeholder engagement and communication skills.

Desirable Skills

• 15+ years' experience in Data Engineering, Data Architecture, or Solution Architecture.
• Experience delivering large-scale enterprise data transformation programs.
• Strong background in cloud migration and analytics modernisation.
• Experience with Azure Event Hubs, Kafka, or streaming technologies.
• Exposure to Microsoft Purview or enterprise data governance platforms.
• Experience supporting AI, machine learning, or advanced analytics workloads.
• Experience within Financial Services, Healthcare, Retail, Logistics, Telecommunications, or Government sectors.
• Azure and Databricks certifications.

Why Join Us?

• Work on large-scale enterprise data transformation initiatives.
• Design and deliver modern Azure Data Lakehouse platforms.
• Influence enterprise data architecture and governance standards.
• Collaborate with experienced cloud, data, and analytics specialists.
• Gain exposure to leading Databricks and Azure technologies.
• Play a key role in shaping strategic data capabilities for major organisations.