Aps6 Senior Sap Data Engineer

Softtest Pays Pty; Canberra; Hybrid; AUD 150,000 - 190,000; Full time

Benefits offered by this job

Hybrid work arrangement

Job summary

INGRITY seeks an experienced Lead Data Engineer to design, build and optimise data ingestion, transformation and visualisation solutions across Azure and Databricks. You will prepare data to support decision‑making across a federal government department, with a strong focus on security and scalable patterns.

The role involves leading data engineering activities, collaborating with cross‑functional teams, and contributing to the Open Data Cube codebase.

Qualifications

• Five years’ data engineering experience with relevant training.
• ETL/ELT delivery in cloud environments, scalable patterns.
• Ability to prepare analytics-ready data in cloud engines.

Responsibilities

• Design, build, test and optimise data ingestion and transformation pipelines.
• Develop reusable ingestion patterns and frameworks.
• Prepare data for analytics, visualisation and AI use cases.
• Provide quality reviews and feedback to data engineers.
• Collaborate with project teams to deliver aligned outcomes.
• Participate in agile ceremonies and team events.

Skills

Data engineering

ETL/ELT

Performance optimization

Automated ingestion

Parquet/Delta formats

Power BI

Communication

Education

Data engineering qualifications

Tools

Azure Databricks

Azure Data Factory

Azure SQL

Spark

Python

Job description

Location:

Canberra CBD, ACT | Hybrid

Engagement:

12-month contract, with two possible 12-month extensions

About INGRITY:

INGRITY is a specialist Data, Digital, Cloud & AI consultancy, helping organisations turn complex data into practical business outcomes.

About the Role:

You’ll lead the design, build and testing of data ingestion, transformation and visualisation solutions across Azure and Databricks, preparing data to support decision-making across the department.

What You’ll Do:

• Design, build, test and maintain data ingestion and transformation solutions that meet current and emerging needs.
• Develop reusable, automated ingestion and transformation patterns and frameworks.
• Prepare data for analytics, visualisation and AI use cases, with a focus on quality and query performance.
• Support data engineers across delivery teams through quality reviews and constructive feedback.
• Collaborate with project teams to deliver outcomes aligned with departmental and program priorities.
• Participate in agile delivery, including in-person attendance at quarterly agile ceremonies and key team events.

What We’re Looking For:

• Data engineering experience: At least five years’ experience as a data engineer, supported by relevant qualifications or training.
• ETL and ELT delivery: Strong experience developing solutions for large or complex data movement, transformation and visualisation, particularly in cloud environments.
• Performance optimisation: Experience preparing data for efficient querying in cloud compute engines, including Spark, graph databases and Azure SQL.
• Automated ingestion: Experience sourcing and ingesting data from Oracle, Ingres, Azure and SQL Server environments.
• Analytics‑ready data: Experience transforming data into Parquet and Delta formats for advanced analytics and AI use cases.
• Power BI: Ability to analyse complex datasets and design visualisations suited to the intended audience.
• Communication and collaboration: Strong written and verbal communication, relationship‑building and negotiation skills.

Nice to Have:

• Experience using Azure Data Factory and Databricks together.
• Knowledge of data lake and lakehouse architectures.
• Experience with ArcGIS Enterprise.

Why Join:

• Deliver data engineering solutions that support decision‑making across a federal government department.
• Work across Azure and Databricks within an established agile team.
• Combine hands‑on engineering with technical leadership and quality oversight.
• Secure a 12‑month engagement with the possibility of two further 12‑month extensions.

Citizenship and Security Clearance:

Applicants must be Australian citizens and hold, or be eligible and willing to obtain, Negative Vetting Level 1 (NV1) clearance.

About the Client:

Our client is a Federal Government organisation delivering nationally significant scientific data, digital infrastructure and earth observation capabilities. Their work supports government, academia and industry through access to satellite imagery, spatial data and digital tools that help identify and understand physical changes across Australia and the Indo‑Pacific.

They are seeking an experienced Lead Data Engineer to join a specialist Data Operations team supporting the maintenance and delivery of scientific products and infrastructure capabilities.

As the Lead Data Engineer, you will work across data management, IT systems and earth observation product support, providing both hands‑on engineering expertise and technical leadership.

The position has a strong focus on cloud‑based data engineering, orchestration, containerisation and Python development, while ensuring security is embedded throughout solutions and delivery pipelines.

Key Responsibilities:

• Designing, developing, validating, maintaining and optimising machine learning workflows through orchestration, testing and documentation.
• Supporting the development and maintenance of data sourcing, ingestion and transformation pipelines.
• Contributing to the development, testing and maintenance of the Open Data Cube code base.
• Implementing, maintaining and improving automated testing and deployment pipelines within AWS environments.
• Providing technical leadership across software development and championing secure, high‑quality engineering practices, tools and standards.
• Supporting knowledge transfer and upskilling of Earth Observation Scientists and providing guidance on improvements to existing algorithms.
• Embedding security tools and practices within systems and CI/CD pipelines.
• Helping ensure services and components remain available and high performing.
• Working collaboratively across technical teams while contributing to a strong, collaborative engineering culture.

Technical Environment:

The technical environment may include AWS, Kubernetes, Terraform, Python, PostgreSQL, Flyte, Airflow, Argo, data processing pipelines, big data technologies and GIS software. Candidates are not expected to have experience across every technology listed.

Required Skills:

• Performance Data Steps Project Delivery Support Database ETL Development Collaboration Analysis Reviews Workshops Developers Continuous Improvement Integration Documentation Maintenance Design SQL Training Leadership

Google Cloud Opportunity:

We’re looking for an experienced Cloud Pak for Data Engineer to support the installation, automation and upgrade of Cloud Pak for Data environments running on Red Hat OpenShift.

Key Responsibilities:

• Install and configure Cloud Pak for Data 5.3.1.
• Enable Helm chart‑based deployments.
• Use cpd-cli to generate image digests and service‑specific CASE files.
• Design and implement GitOps pipelines in GitLab to automate CPD upgrades from version 5.3.1 to 5.4.x.
• Develop and maintain automation scripts that improve operational efficiency and system functionality.
• Investigate and resolve issues across CPD, Red Hat OpenShift and deployment automation.
• Produce clear technical documentation and operational runbooks.

Required Skills and Experience:

• Demonstrated experience with Cloud Pak for Data, including IKC and Manta.
• Strong experience working with Red Hat OpenShift.
• Sound knowledge of Helm charts, Kubernetes and GitLab CI/CD.
• Hands‑on experience with cpd-cli, CASE packages and CPD platform upgrades.
• Scripting experience using Bash, Python or a similar language.
• Experience implementing GitOps practices and automated deployment solutions.

Security Requirements:

Applicants must hold a minimum Australian Government Baseline security clearance. Candidates with an NV1 clearance will be highly regarded.

Australia Data Engineer (El1):

Meaningful brief role description for an EL1 Data Acquisition Engineer in the NDIA data warehouse platform, focusing on technical leadership, data delivery framework, and system documentation.

Key Duties and Responsibilities:

• Provide technical leadership to the project delivery team through design reviews, advising on implementation best practices, defining and refining release process and technical workshops.
• Review and refine the data delivery framework to enhance integration of new sources, performance of existing data delivery and rapid development of new subject areas.
• Own the technical implementation of new data from analysis to delivery (designing and delivering data dictionary, defining the release steps, defining best practices of data modelling and review steps).
• Facilitate continuous improvement measures in delivery principles, coding standards, documentation and provide training sessions to team.
• Provide knowledge transfer sessions and peer review on best practices on development, data modelling, release process etc.
• Prioritise work items and add them to a work queue.
• Understand, analyse and size user requirements.
• Development and maintenance of SQL analytical and ETL code.
• Development and maintenance of system documentation.
• Collaboration with data consumers, database development, testers and IT support teams.

Optional Responsibilities:

NOTE: the key responsibilities of the role are based on current priorities and may change over time.

Job Details (El1):

Role/s: EL1 Data Acquisition Engineer.

The EL1 Data Engineer (Labour Hire) role will be a senior member of a project stream of the NDIA data warehouse platform. This includes updating data development documentation, hands‑on data platform development, providing technical guidance to developers and reviewing/approving data designs and data models.

Requirements:

• Bachelor's or Master's degree in Computer Science, Software Engineering or a related field.
• 3+ years of experience in data engineering, software development, or a related field.
• Strong programming skills in Scala, Python, or Java.
• Expertise in Spark, Kafka, and big data processing concepts.
• Experience with data storage solutions, such as Hadoop HDFS or Amazon S3, and data serialization formats.
• Knowledge of database concepts, data modelling, schema design, and normalisation.
• Familiarity with monitoring and logging tools, and workflow management tools, such as Apache Airflow.
• Experience with version control systems, such as Git.

Your attributes and aptitude:

• Strong communication, problem‑solving, and collaboration abilities.
• Ability to work independently and in a team‑oriented, collaborative environment.

Security Clearance:

Required: Baseline (NV1 preferred)

Technical Environment (Cloud Pak for Data):

Location: Canberra, with onsite and remote working arrangements.

Engagement: Approximately 80 days, full‑time.

Security clearance: Minimum Baseline clearance, with NV1 preferred.

About the Opportunity:

A technology consultancy is seeking an experienced Cloud Pak for Data Engineer to support the installation, automation and upgrade of Cloud Pak for Data environments running on Red Hat OpenShift.

The hands‑on role will focus on configuring CPD 5.3.1, developing automated deployment processes and implementing GitOps pipelines to support upgrades to CPD 5.4.x. The successful candidate will also troubleshoot platform and automation issues and produce supporting technical documentation and operational runbooks.

Offer:

• A hands‑on engagement working with Cloud Pak for Data and Red Hat OpenShift.
• The opportunity to deliver a defined CPD upgrade and deployment automation program.
• Exposure to Helm, GitLab CI/CD, GitOps, Kubernetes and platform automation.
• A Canberra‑based engagement offering onsite and remote working arrangements.

Applications open until 8am 13 November 2026.