Data Products Architect: Azure Databricks Platform Lead

Australian Prudential Regulation Authority; Sydney; On-site; AUD 180,000 - 250,000; Full time

Job summary

Australian Prudential Regulation Authority (APRA) seeks an experienced Data Products Solution Architect to shape APRA’s enterprise data platforms from Sydney CBD NSW. You will steer Azure Databricks initiatives, modernise legacy systems, and lead secure, scalable data solutions while guiding strategy to delivery across data, integration, analytics and AI.

You’ll collaborate with data engineering, enterprise architecture, cyber security and delivery teams to translate architecture into practical

Qualifications

• Experience with enterprise-scale Azure Databricks architecture and delivery.
• Strong knowledge of Unity Catalog, Delta Lake, Workflows, Databricks SQL, serverless compute and lakehouse architecture.
• Practical engineering and DataOps experience across Git, CI/CD, automated testing and observability.
• Strong data modelling and migration experience across legacy data warehouses and modern cloud data platforms.
• Experience designing secure, scalable and resilient enterprise solutions.

Responsibilities

• Lead solution architecture across APRA's data products and platforms, initially focusing on Azure Databricks.
• Define current, transition and target-state architectures and technology roadmaps.
• Lead Databricks architecture across platform design, governance, orchestration, security, resilience, observability, performance and cost.
• Develop solution designs and architecture decisions and guide solutions through governance.
• Define reusable data, integration, engineering and DataOps patterns, including CI/CD, automated testing and infrastructure as code.
• Lead architecture for data migration, coexistence and legacy platform decommissioning.
• Contribute to APRA's broader data products and AI roadmaps.
• Partner with delivery teams to translate architecture into practical implementation and support transition into business-as-usual.
• Provide technical leadership and design assurance to engineers, vendors and senior stakeholders.
• Clearly communicate technology options, risks, dependencies and trade-offs to technical and non-technical audiences.

Skills

Azure Databricks

Databricks SQL

Unity Catalog

Delta Lake

Data governance

CI/CD

Observability

Security

Stakeholder management

Power BI

MuleSoft

Tools

Power BI

MuleSoft

APIs

Job description

Australian Prudential Regulation Authority (APRA) seeks an experienced Data Products Solution Architect to shape APRA’s enterprise data platforms from Sydney CBD NSW. You will steer Azure Databricks initiatives, modernise legacy systems, and lead secure, scalable data solutions while guiding strategy to delivery across data, integration, analytics and AI.

You’ll collaborate with data engineering, enterprise architecture, cyber security and delivery teams to translate architecture into practical