Data Solution Architect - Databricks
Showtime Consulting Pty Ltd
Get more replies from employers
Send a job-specific resume in minutes.
Job summary
Showtime Consulting Pty Ltd is seeking an experienced Senior Data Engineer / Data Architect to join our high‑performing data and cloud engineering team. You will design and deliver enterprise Data Lakehouse solutions on Azure Databricks, modernising legacy platforms and enabling cloud-first analytics across business units.
The role requires deep expertise in PySpark, Delta Lake, Azure Data Services, and data governance, with a track record of large-scale data transformations.
Qualifications
- Experience as a Data Engineer, Data Architect, or similar role.
- Strong hands-on experience with Azure Databricks.
- Expertise with PySpark, Spark SQL, and Delta Lake.
- Experience with Azure Data Factory (ADF), ADLS, and Azure Synapse Analytics.
- Strong data modelling experience, including Dimensional Modelling and Data Vault.
- Deep understanding of data governance, data architecture, security, and data quality.
- Experience delivering real-time and streaming data solutions.
- Strong Azure cloud platform experience.
- Experience with Git, CI/CD, and DevOps practices.
- Excellent stakeholder engagement and communication skills.
Responsibilities
- Design and implement enterprise Data Lakehouse solutions using Azure Databricks.
- Lead data modernisation and cloud migration initiatives.
- Build scalable batch and real-time data processing solutions.
- Develop and maintain data pipelines using PySpark and Spark SQL.
- Define data governance, security, lineage, and data quality standards.
- Design dimensional and Data Vault data models.
- Collaborate with business and technology stakeholders to deliver strategic data platforms.
- Drive data architecture standards, governance, and technical governance.
- Implement CI/CD and DevOps across data engineering workloads.
- Provide technical leadership and mentor engineering teams.
Skills
Tools
Job description
At Showtime Consulting, we don’t just build ultra-secure cloud solutions—we enable organisations to thrive in the most challenging high-threat environments. If you’re truly passionate about innovation, security, and making a real impact, you’re in the right place.
We deliver security where it matters most—protecting national infrastructure, defence, and mission-critical operations. Our solutions are trusted by leading organisations, and our team is at the forefront of innovation in ultra-secure cloud and cross-domain security.
At Showtime Consulting, we don’t just build ultra-secure cloud solutions—we enable organisations to thrive in the most challenging high-threat environments. If you’re truly passionate about innovation, security, and making a real impact, you’re in the right place.
We deliver security where it matters most—protecting national infrastructure, defence, and mission-critical operations. Our solutions are trusted by leading organisations, and our team is at the forefront of innovation in ultra-secure cloud and cross-domain security.
Our people are specialists in secure cloud, defence technology, and insider risk mitigation. We thrive in high-stakes environments, where cybersecurity isn’t just a requirement—it’s mission-critical. If you’re ready to contribute to industry-leading projects and grow in a company that values trust, innovation, and resilience, explore our open roles today.
Principal Cloud Consultant
Resource and Operations Manager at Showtime Consulting
People Operations Lead
Associate Consultant at Showtime Consulting Bachelor of Information Technology Student
CEO at Showtime Consulting | Expert in Ultra-Secure Cloud, Cross-Domain Solutions, Insider Risk Management & Defence-Grade Cybersecurity
Explore Our Current Opportunities And Find A Role That Matches Your Expertise.
Company Description
Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, data, DevSecOps, platform engineering, AI, and digital transformation programs.
We partner with major organisations to design, modernise, and scale data platforms that enable advanced analytics, operational intelligence, regulatory compliance, and data-driven decision making.
Our teams work at the forefront of cloud data engineering and modern analytics, helping customers transform legacy data estates into scalable, cloud-native platforms built for the future.
The Role
We are seeking an experienced Senior Data Engineer / Data Architect to join a high-performing data and cloud engineering team.
This is a hands‑on role focused on designing and delivering enterprise-scale Data Lakehouse solutions using Azure Databricks, modernising legacy data platforms, and enabling cloud‑first analytics capabilities. You will play a key role in defining data architecture, implementing modern data engineering practices, and delivering strategic data platforms that support business-critical reporting, analytics, machine learning, and operational workloads.
The ideal candidate will bring deep expertise across Azure Databricks, PySpark, Delta Lake, Azure Data Services, Data Architecture, and Data Governance, along with a proven track record delivering large-scale data transformation and cloud migration initiatives.
Key Responsibilities
- Design and implement enterprise Data Lakehouse solutions using Azure Databricks.
- Lead data modernisation and cloud migration initiatives.
- Build scalable batch and real-time data processing solutions.
- Develop and maintain data pipelines using PySpark and Spark SQL.
- Define data governance, security, lineage, and data quality standards.
- Design dimensional and Data Vault data models.
- Collaborate with business and technology stakeholders to deliver strategic data platforms.
- Drive data architecture standards, best practices, and technical governance.
- Implement CI/CD and DevOps practices across data engineering workloads.
- Provide technical leadership and mentor engineering teams.
What You'll Need
- Experience as a Data Engineer, Data Architect, or similar role.
- Strong hands‑on experience with Azure Databricks.
- Expertise with PySpark, Spark SQL, and Delta Lake.
- Experience with Azure Data Factory (ADF), ADLS, and Azure Synapse Analytics.
- Strong data modelling experience, including Dimensional Modelling and Data Vault.
- Deep understanding of data governance, data architecture, security, and data quality.
- Experience delivering real-time and streaming data solutions.
- Strong Azure cloud platform experience.
- Experience with Git, CI/CD, and DevOps practices.
- Excellent stakeholder engagement and communication skills.
Desirable Skills
- 15+ years' experience in Data Engineering, Data Architecture, or Solution Architecture.
- Experience delivering large-scale enterprise data transformation programs.
- Strong background in cloud migration and analytics modernisation.
- Experience with Azure Event Hubs, Kafka, or streaming technologies.
- Exposure to Microsoft Purview or enterprise data governance platforms.
- Experience supporting AI, machine learning, or advanced analytics workloads.
- Experience within Financial Services, Healthcare, Retail, Logistics, Telecommunications, or Government sectors.
- Azure and Databricks certifications.
Why Join Us?
- Work on large-scale enterprise data transformation initiatives.
- Design and deliver modern Azure Data Lakehouse platforms.
- Influence enterprise data architecture and governance standards.
- Collaborate with experienced cloud, data, and analytics specialists.
- Gain exposure to leading Databricks and Azure technologies.
- Play a key role in shaping strategic data capabilities for major organisations.
Build solutions that protect defence, national security, and enterprise clients.
Attractive salary packages, incentives, and professional development support.
A collaborative, high-trust workplace where innovation thrives
Access to cybersecurity certifications, mentoring, and growth opportunities.
Remote-friendly roles and adaptable work environments.
Work on Mission-Critical Projects
Build solutions that protect defence, national security, and enterprise clients.
Flexibility & Work-Life Balance
Remote-friendly roles and adaptable work environments.
Access to cybersecurity certifications, mentoring, and growth opportunities.
Attractive salary packages, incentives, and professional development support.
Strong Team Culture
A collaborative, high-trust workplace where innovation thrives
Essential for protecting sensitive information and ensuring the company's technology cannot be compromised.
Critical for building trust with stakeholders and maintaining ethical standards in all operations, especially in such sensitive sectors.
Necessary for staying ahead of technological advancements and threats, ensuring solutions are cutting‑edge and effective.
Demonstrates dedication to the mission, the safety and security of clients’ data, and the success of their operations.
Important for providing reliable and consistent service, which is crucial for defense and intelligence operations that depend on continuity and dependability.
Fuels the drive to excel, innovate, and overcome challenges. It's the emotional investment in the mission and the work, which can inspire teams and lead to exceptional outcomes.
Essential for protecting sensitive information and ensuring the company's technology cannot be compromised.
Critical for building trust with stakeholders and maintaining ethical standards in all operations, especially in such sensitive sectors.
Necessary for staying ahead of technological advancements and threats, ensuring solutions are cutting‑edge and effective.
Demonstrates dedication to the mission, the safety and security of clients’ data, and the success of their operations.
Important for providing reliable and consistent service, which is crucial for defense and intelligence operations that depend on continuity and dependability.
Fuels the drive to excel, innovate, and overcome challenges. It's the emotional investment in the mission and the work, which can inspire teams and lead to exceptional outcomes.
Protecting Operations in Hostile and Sensitive Environments
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
