Senior Data Engineer, Databricks Lakehouse & AI Platform

WRKR

Sydney

Vor Ort

AUD 160.000 - 180.000

Vollzeit

vor 3 Stunden
Sei unter den ersten Bewerbenden

Erhalte eine Antwort von diesem Arbeitgeber

Dein Lebenslauf und dein Anschreiben, genau auf das zugeschnitten, was gesucht wird.

Schaffe es an den ATS-Filtern vorbei

Benefits dieser Stelle

Superannuation
Learning & development
Architecture influence

Zusammenfassung

Wrkr Ltd in Australia seeks a senior Data Engineer to lead the data lakehouse on Databricks in Adelaide. You will design, build and run scalable data pipelines, govern data assets, and drive data quality, with a strong emphasis on security and governance for reporting and AI initiatives.

You will mentor engineers, shape the technical direction with the Head of Data & AI, and contribute to architecture decisions while delivering customer-ready data products and analytics capabilities.

Qualifikationen

  • 5+ years as a Data Engineer or similar, including senior level.
  • Strong Python, PySpark and SQL for production pipelines.
  • Hands-on Databricks experience: Delta Lake, Unity Catalog, Lakeflow Jobs and Databricks SQL.
  • Solid data modelling skills, including Kimball dimensional modelling and medallion architecture.
  • Experience with CDC, incremental processing and streaming or event-driven sources.
  • A track record of building data quality, testing and observability into pipelines.
  • AWS experience (S3, IAM, Glue, Athena), and strong SQL performance tuning, ideally on PostgreSQL.
  • IaC and CI/CD for data platforms, and Git, including rebase-based workflows.
  • Experience mentoring engineers and contributing to data strategy and architecture decisions.
  • Clear communication, including explaining data concepts to non-technical people.

Aufgaben

  • Design, build and run scalable, secure pipelines in Python, PySpark and SQL across bronze, silver and gold layers for reporting.
  • Ingest data through CDC and lead shift to event-sourced ingestion with a data contract per service.
  • Own data quality and freshness through tests, expectations, reconciliations and monitoring.
  • Implement governance and access control in Unity Catalog, including lineage and safe handling of PII.
  • Deliver data products to customers via Delta Sharing and to internal users via Databricks SQL and QuickSight.
  • Manage workspaces, jobs and AWS resources with Terraform and GitLab pipelines.
  • Optimize performance and cost of Spark workloads, SQL warehouses and serverless compute.
  • Prepare analysis-ready datasets for AI and machine learning use cases.
  • Mentor one to two engineers through pairing, code review, 1:1s and development plans.
  • Set and champion engineering standards: code review, testing, documentation and ADRs.
  • Contribute to the Data & AI strategy and roadmap, including architecture decisions.

Kenntnisse

Python
PySpark
SQL
Databricks
Delta Lake
Unity Catalog
Kimball modelling
CDC / event-driven
AWS
IaC / CI/CD
Mentoring
Communication

Ausbildung

Bachelor's degree in CS or related field

Tools

Databricks SQL
Terraform
GitLab CI/CD
Delta Sharing

Jobbeschreibung

Wrkr Ltd in Australia seeks a senior Data Engineer to lead the data lakehouse on Databricks in Adelaide. You will design, build and run scalable data pipelines, govern data assets, and drive data quality, with a strong emphasis on security and governance for reporting and AI initiatives.

You will mentor engineers, shape the technical direction with the Head of Data & AI, and contribute to architecture decisions while delivering customer-ready data products and analytics capabilities.

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Senior Azure Databricks Data Engineer (Lakehouse)
Senior Azure Databricks Data Engineer (Lakehouse)

XPT Software Australia Pty Ltd • Sydney

Vor Ort
AUD 180.000 - 230.000
Senior Azure Data Engineer - Databricks & Lakehouse
Senior Azure Data Engineer - Databricks & Lakehouse

XPT Software • Canberra

Vor Ort
AUD 150.000 - 190.000
Senior Data Engineer: AI-Driven Lakehouse & Cloud Pipelines
Senior Data Engineer: AI-Driven Lakehouse & Cloud Pipelines

Mantel Group Pty Ltd • Australien

Vor Ort
AUD 140.000 - 190.000
Databricks Lakehouse Architect & Strategy Lead
Databricks Lakehouse Architect & Strategy Lead

Ncsaustralia • Sydney

Vor Ort
AUD 180.000 - 230.000
Senior Databricks Engineer — AI/Data Platform Leader (Hybrid)
Senior Databricks Engineer — AI/Data Platform Leader (Hybrid)

Opus Recruitment Solutions Limited • Sydney

Hybrid
AUD 150.000 - 190.000
Databricks Data Engineer: Cloud Lakehouse & Pipelines
Databricks Data Engineer: Cloud Lakehouse & Pipelines

Ingrity • Sydney

Hybrid
AUD 120.000 - 180.000
Hybrid work environment
Databricks Data Engineer - AI & Cloud Platform Lead
Databricks Data Engineer - AI & Cloud Platform Lead

Opus Recruitment Solutions Limited • Sydney

Vor Ort
AUD 120.000 - 180.000
Databricks Lakehouse Architect & Enterprise Data Leader
Databricks Lakehouse Architect & Enterprise Data Leader

NCS Australia • Sydney

Vor Ort
AUD 180.000 - 260.000
Senior Solutions Architect - Enterprise Data & AI Strategy
Senior Solutions Architect - Enterprise Data & AI Strategy

Databricks Inc. • Sydney

Vor Ort
AUD 180.000 - 240.000
Databricks Lead Architect for Enterprise Data & Cloud
Databricks Lead Architect for Enterprise Data & Cloud

NCS Australia • Brisbane

Vor Ort
AUD 190.000 - 230.000
Top-Standorte