Data Platform Engineer: Databricks & Lakehouse (Hybrid)

Helia; Sydney; Hybrid; AUD 110,000 - 160,000; Full time

Benefits offered by this job

Hybrid work policy

Learning & development

Well-being allowance

Job summary

Helia in Sydney is seeking a Data Engineer to join a major financial services program in a hybrid Sydney/Brisbane setup. You will work in a modern Databricks environment, building scalable data pipelines, transforming complex datasets and helping migrate legacy data platforms to a cloud-based lakehouse architecture.

You will design and develop data pipelines using Databricks, PySpark and SQL, build transformations with dbt and Delta Lake, and model data across Bronze, Silver and Gold layers.

Qualifications

• Strong hands-on experience with Databricks, PySpark and SQL.
• Experience with dbt, Delta Lake and lake house architecture.
• Proven experience developing and maintaining ETL/ELT pipelines.
• Experience with cloud platforms, ideally Azure or AWS.
• Understanding of CI/CD, data testing and production support.
• Experience in insurance or financial services highly regarded.
• Exposure to AI-assisted development tools such as GitHub Copilot or Databricks Genie advantageous.

Responsibilities

• Design, develop and optimise data pipelines using Databricks, Python/PySpark and SQL.
• Build and maintain data transformations using dbt and Delta Lake.
• Develop scalable data models across Bronze, Silver and Gold layers.
• Support data migration from legacy platforms, including SAS.
• Implement data quality checks, testing and performance optimisation.
• Collaborate with data engineers, analysts and business stakeholders.

Skills

Databricks

PySpark

SQL

dbt

Delta Lake

ETL/ELT

Azure

AWS

CI/CD

Financial services

GitHub Copilot

Databricks Genie

Job description

Helia in Sydney is seeking a Data Engineer to join a major financial services program in a hybrid Sydney/Brisbane setup. You will work in a modern Databricks environment, building scalable data pipelines, transforming complex datasets and helping migrate legacy data platforms to a cloud-based lakehouse architecture.

You will design and develop data pipelines using Databricks, PySpark and SQL, build transformations with dbt and Delta Lake, and model data across Bronze, Silver and Gold layers.