Data Solution Architect(DataBricks)

Zone IT Solutions; Council of the City of Sydney; On-site; AUD 180,000 - 260,000; Full time

Job summary

Zone IT Solutions is seeking a skilled Data Engineer to design and implement enterprise Data Lakehouse architectures using Databricks on Azure. You will lead data modernization and cloud migration efforts, architect scalable batch and real-time data solutions, and establish governance, security, lineage, and quality frameworks.

You will collaborate with stakeholders to deliver strategic data platforms while upholding architecture standards, best practices, and governance across projects.

Qualifications

• Bachelor's degree or higher in CS/IT or related field.
• Azure certifications and data engineering credentials are a plus.
• 15+ years of experience in Data Engineering, Data Architecture, or Solution Architecture.

Responsibilities

• Design and implement enterprise Data Lakehouse architectures using Databricks.
• Lead data modernization and cloud migration initiatives to enhance data capabilities.
• Architect scalable batch and real-time data solutions to meet business needs.
• Define and enforce data governance, security, lineage, and quality frameworks.
• Collaborate with business and technology stakeholders to deliver strategic data platforms.
• Drive architecture standards, best practices, and technical governance across projects.
• Mentor engineering teams and provide technical leadership across programs.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

Azure Data Factory

ADLS

Azure Synapse Analytics

Data Modeling

Data Governance

Real-time processing

Azure Cloud Platform

CI/CD

Git

Education

Bachelor's degree in Computer Science, Information Technology, or related field

Tools

Azure Databricks

Azure Data Factory

Azure Data Lake Storage

Azure Synapse Analytics

Git

CI/CD pipelines

DevOps practices

Job description

We are seeking a skilled Data Engineer with a strong background in designing and implementing enterprise Data Lakehouse architectures using Databricks. The ideal candidate will lead data modernization and cloud migration initiatives, architect scalable batch and real time data solutions, and define data governance, security, lineage, and quality frameworks. This role requires collaboration with business and technology stakeholders to deliver strategic data platforms while driving architecture standards, best practices, and technical governance.

Responsibilities:

• Design and implement enterprise Data Lakehouse architectures utilizing Databricks.
• Lead data modernization and cloud migration initiatives to enhance data capabilities.
• Architect scalable batch and real time data solutions to meet business needs.
• Define and enforce data governance, security, lineage, and quality frameworks.
• Collaborate with business and technology stakeholders to deliver strategic data platforms.
• Drive architecture standards, best practices, and technical governance across projects.
• Mentor engineering teams and provide technical leadership across programs.

Mandatory Skills:

• Proficiency in Azure Databricks.
• Experience with PySpark and Spark SQL.
• Knowledge of Delta Lake.
• Familiarity with Azure Data Factory (ADF).
• Experience with Azure Data Lake Storage (ADLS).
• Understanding of Azure Synapse Analytics.
• Expertise in Data Modeling (Dimensional & Data Vault).
• Strong knowledge of Data Governance and Data Architecture.
• Experience in Real time Data Processing.
• Proficient in Azure Cloud Platform.
• Familiarity with CI/CD, Git, and DevOps practices.

Preferred Skills:

• 15+ years of experience in Data Engineering, Data Architecture, or Solution Architecture.
• Strong background in cloud migration and analytics modernization.
• Experience across industries such as Financial Services, Healthcare, Retail, Logistics, Telecommunications, or Government.

Qualifications:

Bachelor's degree in Computer Science, Information Technology, or a related field. Relevant certifications in Azure and Data Engineering are a plus.