Data Engineer - Databricks

TalentReady; Sydney; Hybrid; AUD 120,000 - 170,000; Full time

Job summary

TalentReady in Australia is seeking an experienced Data Engineer to join a major financial services organisation undergoing data modernisation. You will work within a modern Databricks environment, building scalable data pipelines and supporting the migration of legacy data platforms to a cloud-based lakehouse architecture.

You will collaborate with data engineers, analysts and business stakeholders to design, implement and optimise ETL/ELT processes, with a strong focus on quality and

Qualifications

• Strong hands-on experience with Databricks, PySpark and SQL.
• Experience with dbt, Delta Lake and lakehouse architecture.
• Proven experience developing and maintaining ETL/ELT pipelines.
• Cloud experience, ideally Azure or AWS.
• Understanding of CI/CD, data testing and production support.
• Experience in insurance or financial services highly regarded.

Responsibilities

• Design, develop and optimise data pipelines using Databricks, Python/PySpark and SQL.
• Build and maintain data transformations using dbt and Delta Lake.
• Develop scalable data models across Bronze, Silver and Gold layers.
• Support data migration from legacy platforms, including SAS.
• Implement data quality checks, testing and performance optimisation.
• Collaborate with data engineers, analysts and business stakeholders.

Skills

Databricks

PySpark

SQL

dbt

Delta Lake

Lakehouse

ETL/ELT

Azure

AWS

CI/CD

Tools

GitHub Copilot

Databricks Genie

Job description

Location: Sydney / Brisbane (Hybrid)
Contract: 12 Months
Industry: Financial Services / Insurance

Data Engineer - Databricks

Location: Sydney / Brisbane (Hybrid)
Contract: 12 Months
Industry: Financial Services / Insurance

The Opportunity

We are seeking an experienced Data Engineer to join a major financial services organisation undergoing a significant data modernisation program.

You'll work within a modern Databricks environment, building scalable data pipelines, transforming complex datasets and supporting the migration of legacy data platforms to a cloud-based lakehouse architecture.

Key Responsibilities

• Design, develop and optimise data pipelines using Databricks, Python/PySpark and SQL.
• Build and maintain data transformations using dbt and Delta Lake.
• Develop scalable data models across Bronze, Silver and Gold layers.
• Support data migration from legacy platforms, including SAS.
• Implement data quality checks, testing and performance optimisation.
• Collaborate with data engineers, analysts and business stakeholders.

Skills & Experience

• Strong hands-on experience with Databricks, PySpark and SQL.
• Experience with dbt, Delta Lake and lakehouse architecture.
• Proven experience developing and maintaining ETL/ELT pipelines.
• Experience with cloud platforms, ideally Azure or AWS.
• Understanding of CI/CD, data testing and production support.
• Experience in insurance or financial services highly regarded.
• Exposure to AI-assisted development tools such as GitHub Copilot or Databricks Genie advantageous.

Why Join?

• Work on a large-scale data transformation program.
• Modern Databricks and cloud technology environment.
• Collaborative engineering team with hybrid flexibility.
• Opportunity to contribute to AI-enabled data engineering practices.

Additional information

• Long Term Engagement
• Sydney or Brisbane - Hybrid
• Tech Stack: Databricks, Python, SQL