Databricks Data Engineer – Cloud Lakehouse Pipelines

Ingrity; Sydney, Brisbane; Hybrid; AUD 120,000 - 180,000; Full time

Job summary

INGRITY is seeking an experienced Data Engineer to join a major financial services organisation undergoing a data modernisation program. The role sits in a modern Databricks environment, building scalable pipelines and migrating legacy platforms to a cloud-based lakehouse.

You will design, develop and optimize pipelines, transform complex datasets and collaborate with engineers, analysts, and stakeholders in a hybrid Sydney/Brisbane setting.

Qualifications

• Strong hands-on experience with Databricks, PySpark and SQL.
• Experience with dbt, Delta Lake and lake house architecture.
• Proven experience developing and maintaining ETL/ELT pipelines.
• Experience with cloud platforms, ideally Azure or AWS.
• Understanding of CI/CD, data testing and production support.
• Experience in insurance or financial services highly regarded.
• Exposure to AI-assisted development tools such as GitHub Copilot or Databricks Genie advantageous.

Responsibilities

• Design, develop and optimise data pipelines using Databricks, Python/PySpark and SQL.
• Build and maintain data transformations using dbt and Delta Lake.
• Develop scalable data models across Bronze, Silver and Gold layers.
• Support data migration from legacy platforms, including SAS.
• Implement data quality checks, testing and performance optimisation.
• Collaborate with data engineers, analysts and business stakeholders.

Skills

Databricks

PySpark

SQL

Tools

dbt

Delta Lake

Job description

INGRITY is seeking an experienced Data Engineer to join a major financial services organisation undergoing a data modernisation program. The role sits in a modern Databricks environment, building scalable pipelines and migrating legacy platforms to a cloud-based lakehouse.

You will design, develop and optimize pipelines, transform complex datasets and collaborate with engineers, analysts, and stakeholders in a hybrid Sydney/Brisbane setting.