Senior Data Engineer: Cloud Data Platform & Spark

XPT Software; City of Brisbane; On-site; AUD 120,000 - 160,000; Full time

Job summary

XPT Software in Australia is seeking an experienced Senior Data Engineer to join our Brisbane team. You will design and implement data pipelines, profiling, and cloud-based data platforms using Azure, Databricks, Spark, and related tools.

You will collaborate with cross-location teams, manage structured and unstructured data, and optimize data workflows with Airflow and scheduling tools. Strong SQL and Python, plus cloud and big data experience, are essential.

Qualifications

• Minimum 8 years of experience in data engineering or analytics.
• Strong SQL and Python expertise.
• Experience with data profiling and cloud data warehouses.
• Hands-on with Azure Cloud and Databricks.
• Experience with scheduling tools like Airflow or Control-M.
• Familiarity with big data tools: Spark, Hive, Impala, Hue, HDFS, HBase.
• Experience managing structured and unstructured data.

Skills

Advanced SQL

Python

Data profiling

Azure Cloud

Databricks

Airflow

Control-M

Spark (Python)

Hive

Impala

Hue

HDFS

HBase

Structured data

Unstructured data

Job description

XPT Software in Australia is seeking an experienced Senior Data Engineer to join our Brisbane team. You will design and implement data pipelines, profiling, and cloud-based data platforms using Azure, Databricks, Spark, and related tools.

You will collaborate with cross-location teams, manage structured and unstructured data, and optimize data workflows with Airflow and scheduling tools. Strong SQL and Python, plus cloud and big data experience, are essential.