Data Engineer: Azure Databricks & Data Platform Expert

Colonial First State; Sydney; On-site; AUD 120,000 - 150,000; Full time

Benefits offered by this job

Birthday leave

Employer superannuation

Life leave 3 days a year

Mental health support app (Sonder)

Health insurance with BUPA

Counselling services

Job summary

Colonial First State (CFS) is seeking a Data Engineer within the AI & Data division in Sydney. You will design, build and operate scalable data products, focusing on high availability, performance and security of data environments.

Role requires strong Python, PySpark and SQL skills, and experience with Azure Databricks, ADF, Unity Catalog and Terraform to manage CI/CD pipelines and data pipelines. This is a hands-on role within a high-impact data team.

Qualifications

• Proven experience as a data engineer with scalable, secure data products.
• Experience with cloud-based data platforms and governance.
• Knowledge of Databricks and Azure data services is essential.

Responsibilities

• Provide technical solutions to enhance data engineering capabilities.
• Ensure high availability, performance, and security of data environments.
• Develop and productionize code artefacts; maintain documentation and tests.
• Collaborate with project teams to review code changes and adhere to standards.
• Deliver data warehouse solutions aligned with stakeholder needs.

Skills

Data engineering

Azure Databricks

Python

PySpark

SQL

CI/CD pipelines

Terraform

Databricks Unity Catalog

Azure Data Factory

ADLS

Security controls

Data governance

Data model implementation

Data orchestration

Cloud

Tools

Azure DevOps

GitHub

Terraform

Job description

Colonial First State (CFS) is seeking a Data Engineer within the AI & Data division in Sydney. You will design, build and operate scalable data products, focusing on high availability, performance and security of data environments.

Role requires strong Python, PySpark and SQL skills, and experience with Azure Databricks, ADF, Unity Catalog and Terraform to manage CI/CD pipelines and data pipelines. This is a hands-on role within a high-impact data team.