Data Engineer with NV1 Clearance – Cloudera, Spark & Airflow ETL

Xpt Software Australia Pty; Melbourne; On-site; AUD 120,000 - 180,000; Full time

Job summary

Xpt Software Australia Pty is seeking a Data Engineer to design and optimize ETL pipelines using Spark, NiFi, and Airflow within Cloudera Data Platform. You will configure CDP environments, manage security and access controls, and work with enterprise identity services such as EntraID.

Desirable experience includes Cloudera Data Visualization, MSSQL & PowerBI integration, and prior Defence sector exposure. Strong problem solving and communication skills are essential for cross-team collaboration.

Qualifications

• Must have hands-on ETL development experience in Spark / Cloudera Data Platform.
• Strong experience with NiFi for ETL workflows within CDP environments.
• Proficient in DDLs and Erwin for data modeling and lineage.
• Experience with Ozone & Iceberg, Hive, Hue and Impala in data pipelines.
• Ability to configure and manage CDP environments and access controls, including enterprise identity services (e.g., EntraID).
• DevOps skills using AzureDevOps / VisualStudio Code are essential.
• Expertise in debugging complex ETL pipelines and various ETL methods (truncate & reload, etc.).
• Experience with Cloudera Data Visualization is a plus.
• Experience with MSSQL and PowerBI is a plus.
• Excellent problem solving, communication and task management skills.
• Past work on large agency data warehouses and data integration projects is required.
• Defence sector experience is desirable.

Responsibilities

• Develop and optimize ETL pipelines using Spark, NiFi and Airflow within Cloudera Data Platform.
• Manage CDP environments, security roles and access controls linked to identity services.
• Debug and design robust ETL methods including truncate & reload, rollback, and auditing.
• Collaborate with data visualisation efforts using Cloudera Data Visualisation, MSSQL and PowerBI when needed.
• Communicate clearly with stakeholders and manage tasks across data engineering projects.
• Leverage DevOps practices via AzureDevOps and Visual Studio Code to streamline deployments.

Skills

ETL pipeline development

Spark

Cloudera Data Platform

Airflow

NiFi

DDL

Erwin

Ozone

Iceberg

Hive

Impal a

Ranger security

Atlas security

EntraID

AzureDevOps

VisualStudio Code

ETL debugging

Cloudera Data Visualization

MSSQL

PowerBI

Problem solving

Communication

Task management

Defence experience

Tools

NiFi

AzureDevOps

VisualStudioCode

PowerBI

MSSQL

Cloudera Data Platform

Airflow

Spark

Job description

Xpt Software Australia Pty is seeking a Data Engineer to design and optimize ETL pipelines using Spark, NiFi, and Airflow within Cloudera Data Platform. You will configure CDP environments, manage security and access controls, and work with enterprise identity services such as EntraID.

Desirable experience includes Cloudera Data Visualization, MSSQL & PowerBI integration, and prior Defence sector exposure. Strong problem solving and communication skills are essential for cross-team collaboration.