Data Engineer- NV1 Security Clearance

XPT Software Australia Pty Ltd; Melbourne, Canberra, Brisbane; On-site; AUD 120,000 - 180,000; Full time

Job summary

XPT Software Australia Pty Ltd is seeking a Data Engineer with NV1 security clearance to join our Melbourne team. The role involves developing ETL pipelines in Spark within Cloudera Data Platform, using NiFi, and ensuring robust data security with Ranger/Atlas.

Experience in Hive/Impala, Erwin data modeling, and enterprise identity integrations is required. Defence-domain experience is desirable; strong problem solving and collaboration are essential.

Qualifications

• NV1 security clearance required.
• ETL development in Spark within Cloudera Data Platform.
• NiFi ETL development and Erwin data modeling.
• ETL using Ozone & Iceberg, Hive, Hue and Impala.
• Config and management of CDP environments including Ranger/Atlas security.
• Complex ETL pipeline debugging and design (truncate & reload, etc).
• Cloudera Data Visualisation is a nice-to-have.

Responsibilities

• Develop and maintain ETL pipelines using Spark and NiFi within CDP.
• Configure and secure CDP environments with Enterprise identity services.
• Debug complex ETL pipelines and implement various data loading methods.
• Collaborate on data warehousing and defence-related data integration projects.
• Explore MSSQL and PowerBI integration as optional enhancements.

Skills

Spark

Airflow

ETL design

Problem solving

Tools

NiFi

Cloudera Data Platform

Hive

Impala

Erwin

Hue

Ranger

Atlas

PowerBI

MSSQL

Job description

XPT Software Australia Pty Ltd | Contract

Data Engineer- NV1 Security Clearance

Melbourne, Canberra, Brisbane, Australia

• XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company
  
• XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.
  
• We have 120+technocrats in Australia working at our clientlocations.
  
• XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.
  
• We have served100+ clients globally, fulfilling their onsite-offshoreneeds.
  

Job Description

Security clearance: Candidates must hold NV1 clearance

ETL pipeline development in Spark / Cloudera Data Engineering / Airflow (critical requirement)

ETL pipeline development using NiFi within Cloudera Data Platform (strong requirement)

ETL development using DDLs and Erwin (Strong requirement)

ETL development experience using Ozone & Iceberg, Hive, Hue and Impala (critical requirement)

Configuration and management of Cloudera Data Platform environments and tenancies, including complex ranger and atlas security & access controls, linked to enterprise identity management services, such as EntraID

Complex ETL pipeline debugging and design of various methods of ETL, including truncate & reload etc (critical requirement)

Developer / Engineering skills in Cloudera Data Visualisation (nice to have)

Developer / Engineering / Integration skills in MSSQL & PowerBI (nice to have)

Excellent problem solving, communication and task management skills (strong requirement)

Previous experience working on large agency data warehouses and data integration projects (strong requirement)

Pervious experience working in Defence (desirable)