Data Analyst and Engineers

Calleo; Canberra; On-site; AUD 120,000 - 170,000; Full time

Job summary

Calleo is seeking a Senior Data Engineer for a Federal Government project in Canberra. You will migrate SAS code and workflows to Databricks, preserving business rules and data quality, while building scalable ETL/ELT pipelines.

Responsibilities include implementing security and governance controls, and delivering well-documented, maintainable data solutions using PySpark, Spark SQL and Delta Lake.

Qualifications

• Experience designing scalable Databricks data solutions using PySpark, Spark SQL and Delta Lake.
• Experience migrating SAS code to Databricks while preserving logic and data quality.
• Proven ability to build maintainable ETL/ELT pipelines with validation, monitoring and tuning.

Responsibilities

• Migrate SAS code, workflows and reporting to Databricks while preserving business logic.
• Build and optimise ETL/ELT pipelines and data models with validation and monitoring.
• Apply security, privacy, metadata and data governance controls and document work.

Skills

Databricks

PySpark

Spark SQL

SQL

Delta Lake

ETL/ELT pipelines

Data governance

Cloud platforms

Tools

Azure DevOps

Git

CI/CD

Job description

Job Overview

Our Federal Government client is seeking experienced Senior Data Engineers with strong expertise in modern data engineering and cloud-based data platforms.

The successful candidates will have demonstrated experience in migrating SAS code, workflows, and reporting processes to Databricks, while preserving existing business logic and ensuring data integrity, accuracy, and quality throughout the migration process.

Job Requirement

• Demonstrated experience designing and delivering scalable Databricks data solutions using PySpark, Spark SQL, SQL and Delta Lake.
• Demonstrated experience migrating SAS code, workflows and reporting processes to Databricks while preserving business logic and data quality.
• Experience building maintainable ETL/ELT pipelines and data models, including automated validation, monitoring and performance optimisation.
• Proven capability with Azure DevOps, Git, CI/CD, source control and controlled deployment practices.
• Ability to apply security, privacy, metadata and data governance controls and produce clear technical documentation.

Job Location

Canberra

Due to the nature of this role, this job is open to Australian Citizens only. Candidates who currently hold or able to obtain Federal Government Security Clearances are highly desirable.

Calleo is an equal opportunity employer, and we encourage applications from all people including Aboriginal and Torres Strait Islander people.