Azure Databricks Data Engineer — Pipelines & Lakehouse

Ingrity; Sydney; On-site; AUD 110,000 - 160,000; Full time

Job summary

INGRITY is seeking a hands-on Data Engineer in Sydney to design and maintain scalable data pipelines on Azure Databricks and Data Factory. You will deliver PySpark-based solutions, implement ETL/ELT workflows, and support Lakehouse architectures alongside a team of data professionals.

The role emphasizes collaboration with architects and stakeholders, strong data quality standards and a focus on secure, well-documented engineering practices.

Qualifications

• 3–6 years’ experience in Data Engineering or a similar hands-on data role.
• Commercial experience with Azure Databricks.
• Strong experience with Azure Data Factory (ADF).
• Hands-on capability with Python, PySpark, Spark SQL and SQL.
• Experience building ETL/ELT data pipelines.
• Understanding of Lakehouse and Medallion Architecture.
• Experience with Azure data services (ADLS, Azure SQL, Blob Storage, Key Vault).
• Familiarity with Git, Azure DevOps and CI/CD pipelines.
• Understanding of data modelling, data quality and performance optimisation.
• Strong troubleshooting and problem solving; clear communication with stakeholders.

Responsibilities

• Design, develop and maintain scalable data pipelines using Azure Databricks and Azure Data Factory.
• Build and optimise ETL/ELT workflows across cloud platforms.
• Develop solutions using PySpark, Spark SQL, Python and SQL.
• Work with Databricks notebooks, jobs, clusters and workflows.
• Support Lakehouse and Medallion Architecture patterns.
• Collaborate with Data Engineers, Architects and business stakeholders.
• Maintain data quality, security, documentation and engineering best practices.

Skills

Azure Databricks

Azure Data Factory

Python

PySpark

Spark SQL

SQL

Data pipelines

Git

Azure DevOps

CI/CD

Data quality

Troubleshooting

Communication

Tools

dbt

Terraform

Job description

INGRITY is seeking a hands-on Data Engineer in Sydney to design and maintain scalable data pipelines on Azure Databricks and Data Factory. You will deliver PySpark-based solutions, implement ETL/ELT workflows, and support Lakehouse architectures alongside a team of data professionals.

The role emphasizes collaboration with architects and stakeholders, strong data quality standards and a focus on secure, well-documented engineering practices.