Lead Data Platform Engineer - Azure Databricks (Hybrid)

shift

North Sydney

Hybrid

AUD 150,000 - 190,000

Full time

9 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Hybrid working
Parental leave: up to 26 weeks
Strategy Week
Flat structure
Wellbeing benefits

Job summary

Shift is seeking a senior data engineer in Australia to own the Azure Databricks platform, set standards and roadmap, and lead the data workstream of a core lending system migration. You’ll be the senior engineer in a small team, with two other engineers to guide and shape how data works at Shift.

You’ll design robust ELT pipelines, build a scalable dimensional model, and lift data quality and governance while keeping costs efficient.

Qualifications

  • 7+ years in data engineering with hands-on Azure Databricks experience.
  • Strong Python and SQL with pipelines-as-code experience.
  • Proven ability to lift standards on an existing platform.

Responsibilities

  • Own data platform roadmap, standards, CI/CD and governance.
  • Design and lead scalable ELT pipelines on Azure Databricks.
  • Lead data workstream for core system migration and remap pipelines.
  • Develop a dimensional data model and ensure data quality.
  • Mentor other data engineers and drive technical decisions.
  • Collaborate with analysts, data scientists, product managers, and Engineering.

Skills

Azure Databricks
Python
SQL
CI/CD
Azure DevOps
GitHub
Data modelling
Mentoring
Cross-team collaboration

Tools

Delta Lake
Unity Catalog
Databricks Asset Bundles

Job description

Shift is seeking a senior data engineer in Australia to own the Azure Databricks platform, set standards and roadmap, and lead the data workstream of a core lending system migration. You’ll be the senior engineer in a small team, with two other engineers to guide and shape how data works at Shift.

You’ll design robust ELT pipelines, build a scalable dimensional model, and lift data quality and governance while keeping costs efficient.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Platform Engineer — Azure Databricks, Hybrid
Lead Data Platform Engineer — Azure Databricks, Hybrid

Shift • Sydney

Hybrid
AUD 150,000 - 190,000
Hybrid work policy
Parental leave up to 26 weeks
Strategy Week
+2
Lead Data Engineer, Azure Databricks Platform (Hybrid)
Lead Data Engineer, Azure Databricks Platform (Hybrid)

Shift Financial Pty • Sydney

Hybrid
AUD 150,000 - 190,000
Hybrid working
Parental leave
Strategy Week
+2
Senior Data Engineer - Azure Databricks Platform Lead
Senior Data Engineer - Azure Databricks Platform Lead

Shift Financial • Sydney

Hybrid
AUD 140,000 - 190,000
Parental leave
Hybrid working
Strategy Week
+2
Azure Databricks Lead Data Engineer - Hybrid Sydney
Azure Databricks Lead Data Engineer - Hybrid Sydney

u&u Recruitment Partners • Sydney

Hybrid
AUD 190,000 - 240,000
Senior Data Engineer
Senior Data Engineer

Shift Financial Pty • Sydney

Hybrid
AUD 150,000 - 190,000
Hybrid working
Parental leave
Strategy Week
+2
Senior Data Engineer
Senior Data Engineer

Shift Financial • Sydney

Hybrid
AUD 140,000 - 190,000
Parental leave
Hybrid working
Strategy Week
+2
Senior Data Engineer
Senior Data Engineer

Shift • Sydney

Hybrid
AUD 150,000 - 190,000
Hybrid work policy
Parental leave up to 26 weeks
Strategy Week
+2
Senior Data Engineer
Senior Data Engineer

shift • North Sydney

Hybrid
AUD 150,000 - 190,000
Hybrid working
Parental leave: up to 26 weeks
Strategy Week
+2
Senior Data Engineer - Azure Databricks & Lakehouse Lead
Senior Data Engineer - Azure Databricks & Lakehouse Lead

XPT Software • Sydney

On-site
AUD 140,000 - 190,000
Lead Data Engineer: Cloud Data & Databricks Leader (Hybrid)
Lead Data Engineer: Cloud Data & Databricks Leader (Hybrid)

u&u Recruitment Partners • Melbourne

Hybrid
AUD 150,000 - 210,000
Top locations