Senior Data Engineer - Databricks & SAS Migration (Canberra)

Calleo; Canberra; On-site; AUD 120,000 - 170,000; Full time

Job summary

Calleo is seeking a Senior Data Engineer for a Federal Government project in Canberra. You will migrate SAS code and workflows to Databricks, preserving business rules and data quality, while building scalable ETL/ELT pipelines.

Responsibilities include implementing security and governance controls, and delivering well-documented, maintainable data solutions using PySpark, Spark SQL and Delta Lake.

Qualifications

• Experience designing scalable Databricks data solutions using PySpark, Spark SQL and Delta Lake.
• Experience migrating SAS code to Databricks while preserving logic and data quality.
• Proven ability to build maintainable ETL/ELT pipelines with validation, monitoring and tuning.

Responsibilities

• Migrate SAS code, workflows and reporting to Databricks while preserving business logic.
• Build and optimise ETL/ELT pipelines and data models with validation and monitoring.
• Apply security, privacy, metadata and data governance controls and document work.

Skills

Databricks

PySpark

Spark SQL

SQL

Delta Lake

ETL/ELT pipelines

Data governance

Cloud platforms

Tools

Azure DevOps

Git

CI/CD

Job description

Calleo is seeking a Senior Data Engineer for a Federal Government project in Canberra. You will migrate SAS code and workflows to Databricks, preserving business rules and data quality, while building scalable ETL/ELT pipelines.

Responsibilities include implementing security and governance controls, and delivering well-documented, maintainable data solutions using PySpark, Spark SQL and Delta Lake.