Azure Databricks Data Engineer — Hybrid Sydney

Ingrity; Sydney; Hybrid; AUD 120,000 - 150,000; Full time

Job summary

INGRITY is seeking a hands-on Data Engineer in Sydney to design, build and optimize scalable data pipelines on Azure Databricks and Azure Data Factory. You’ll work across the data platform, implementing lakehouse patterns and collaborating with data architects and stakeholders.

The role requires 3–6 years of data engineering experience, strong PySpark/Python/SQL skills, and a solid understanding of CI/CD and cloud data services.

Qualifications

• 3–6 years’ experience in Data Engineering or a similar hands‑on data role.
• Commercial experience with Azure Databricks.
• Strong experience with Azure Data Factory (ADF).
• Good hands‑on capability with Python, PySpark, Spark SQL and SQL.
• Experience building and maintaining ETL/ELT data pipelines.
• Understanding of Lakehouse and Medallion Architecture.
• Experience working with Azure data services such as ADLS, Azure SQL, Blob Storage and Key Vault.
• Familiarity with Git, Azure DevOps and CI/CD pipelines.
• Understanding of data modelling, data quality and performance optimisation.
• Strong troubleshooting and problem‑solving skills.

Responsibilities

• Designing, developing and maintaining scalable data pipelines using Azure Databricks and Azure Data Factory.
• Building and optimising ETL/ELT workflows across cloud-based data platforms.
• Developing solutions using PySpark, Spark SQL, Python and SQL.
• Working with Databricks notebooks, jobs, clusters and workflows.
• Supporting Lakehouse and Medallion Architecture patterns across data platforms.
• Working with Azure services including ADLS/Blob Storage, Azure SQL, Key Vault and Azure DevOps.
• Monitoring data pipelines, troubleshooting issues and performing root‑cause and performance analysis.
• Supporting CI/CD, Git-based development and automated deployments.
• Working closely with Data Engineers, Architects and business stakeholders to deliver reliable data solutions.
• Maintaining strong standards around data quality, security, documentation and engineering best practice.

Skills

Azure Databricks

Python

PySpark

SQL

Data pipelines

ETL/ELT design

CI/CD

Git

Tools

Azure Databricks

Azure Data Factory

Databricks notebooks

Azure DevOps

Terraform

dbt

Job description

INGRITY is seeking a hands-on Data Engineer in Sydney to design, build and optimize scalable data pipelines on Azure Databricks and Azure Data Factory. You’ll work across the data platform, implementing lakehouse patterns and collaborating with data architects and stakeholders.

The role requires 3–6 years of data engineering experience, strong PySpark/Python/SQL skills, and a solid understanding of CI/CD and cloud data services.