Data Engineer- NV1 Security Clearance

XPT Software; Melbourne; On-site; AUD 120,000 - 180,000; Full time

Job summary

XPT Software in Melbourne is seeking a skilled data engineering specialist to lead ETL development on Cloudera Data Platform, Spark and Airflow, with NiFi use on CDP. You will design and optimize pipelines, handle complex data models, and work closely with security teams.

Experience with Ozone/Iceberg, Hive, Hue and Impala, plus strong problem solving and communication, is required. Defence sector experience is desirable; join a team delivering critical infrastructure data solutions in Australia.

Qualifications

• NV1 clearance required.
• ETL development in Spark and Cloudera Data Platform (CDP) with Airflow.
• NiFi within CDP is a strong requirement.
• Proficient with DDLs and Erwin.
• ETL development using Ozone, Iceberg, Hive, Hue and Impala.
• Security and access controls via Ranger/Atlas linked to enterprise identity services such as EntraID.
• Complex ETL debugging and truncate/reload techniques.
• CDP data visualization skills are a plus.
• MS SQL and PowerBI experience are nice to have.
• Strong problem solving, communication and task management; experience in large agency data warehouses and defence is desirable.

Skills

NV1 clearance

Spark ETL

Airflow

NiFi

DDL

Erwin

Ozone

Iceberg

Hive

Hue

Impala

Security & IAM

Tools

Cloudera Data Platform

Ranger

Atlas

EntraID

CDP Data Visualisation

MS SQL

PowerBI

Job description

• Security clearance: Candidates must hold NV1 clearance
• ETL pipeline development in Spark / Cloudera Data Engineering / Airflow (critical requirement)
• ETL pipeline development using NiFi within Cloudera Data Platform (strong requirement)
• ETL development using DDLs and Erwin (Strong requirement)
• ETL development experience using Ozone & Iceberg, Hive, Hue and Impala (critical requirement)
• Configuration and management of Cloudera Data Platform environments and tenancies, including complex ranger and atlas security & access controls, linked to enterprise identity management services, such as EntraID
• Complex ETL pipeline debugging and design of various methods of ETL, including truncate & reload etc (critical requirement)
• Developer / Engineering skills in Cloudera Data Visualisation (nice to have)
• Developer / Engineering / Integration skills in MSSQL & PowerBI (nice to have)
• Excellent problem solving, communication and task management skills (strong requirement)
• Previous experience working on large agency data warehouses and data integration projects (strong requirement)
• Pervious experience working in Defence (desirable)