Data Engineer - Databricks, Delta Lake & Cloud (Contract)

Recruitment Hive; Canberra; On-site; AUD 110,000 - 170,000; Full time

Job summary

The Recruitment Hive is seeking multiple Data Engineers on a 12-month contract to support a newly formed Government Organisation focused on environment, climate change and resources. The role requires Databricks, PySpark and Azure cloud expertise and is based in Canberra with 12-month contracting potential.

Immediate start and a baseline security clearance are preferred. Responsibilities include building enterprise data pipelines, Lakehouse architectures, data ingestion, and quality, with

Qualifications

• Experience integrating data from APIs, databases and SaaS platforms.
• Cloud experience with Azure, ADLS Gen2 and data factories.
• DevOps practices with Git, CI/CD and automated testing.
• Significant Databricks data engineering experience and SQL.
• Strong PySpark, Spark SQL and Python skills.
• Experience with Delta Lake, Unity Catalog and Databricks SQL.

Responsibilities

• Design, develop and maintain enterprise-scale data pipelines using Databricks.
• Build and optimise Lakehouse architectures with Delta Lake and Medallion patterns.
• Develop data ingestion frameworks for structured, semi-structured and unstructured data sources.
• Create scalable data transformation processes using PySpark, Spark SQL and Python.
• Implement data quality, lineage, monitoring and observability capabilities.

Skills

Data integration

Azure cloud

DevOps

Databricks

PySpark / Python

Databricks SQL

Tools

Databricks

Git

Azure Data Factory

Job description

The Recruitment Hive is seeking multiple Data Engineers on a 12-month contract to support a newly formed Government Organisation focused on environment, climate change and resources. The role requires Databricks, PySpark and Azure cloud expertise and is based in Canberra with 12-month contracting potential.

Immediate start and a baseline security clearance are preferred. Responsibilities include building enterprise data pipelines, Lakehouse architectures, data ingestion, and quality, with