Senior Data Engineer

XPT Software; City of Brisbane; On-site; AUD 120,000 - 160,000; Full time

Job summary

XPT Software in Australia is seeking an experienced Senior Data Engineer to join our Brisbane team. You will design and implement data pipelines, profiling, and cloud-based data platforms using Azure, Databricks, Spark, and related tools.

You will collaborate with cross-location teams, manage structured and unstructured data, and optimize data workflows with Airflow and scheduling tools. Strong SQL and Python, plus cloud and big data experience, are essential.

Qualifications

• Minimum 8 years of experience in data engineering or analytics.
• Strong SQL and Python expertise.
• Experience with data profiling and cloud data warehouses.
• Hands-on with Azure Cloud and Databricks.
• Experience with scheduling tools like Airflow or Control-M.
• Familiarity with big data tools: Spark, Hive, Impala, Hue, HDFS, HBase.
• Experience managing structured and unstructured data.

Skills

Advanced SQL

Python

Data profiling

Azure Cloud

Databricks

Airflow

Control-M

Spark (Python)

Hive

Impala

Hue

HDFS

HBase

Structured data

Unstructured data

Job description

• XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company
  
• XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.
  
• We have 120+technocrats in Australia working at our clientlocations.
  
• XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.
  
• We have served100+ clients globally, fulfilling their onsite-offshoreneeds.
  

Minimum Years of Experience: 8Years

Must Have Skills:

· Hands On with advanced SQL, Python etc.

· Hands On in data profiling

· Hands On in working on Cloud like Azure and Cloud DW like Data bricks

· Hands On experience on Scheduling tools like Airflow, Control M etc.

· Knowledgeable on Big Data tools like Spark (Python/Scala), Hive, Impala,Hue and storage (e.g. HDFS, HBase)

· Knowledgeable managing structured and unstructured data types.

Nice to Have Skills:

· Experience in working with a team of data engineers working fromdifferent locations in different time zones.

· Effective prioritization.

· Pragmatic stakeholder management.CloudPlatform Familiarity: Understanding of cloud services (AWS, Azure, GCP) toanalyze and optimize cloud-based systems.

· Basic Scripting Knowledge: Ability to write simple scripts in Python,SQL, or R for data extraction, transformation, and analysis.

Be careful - Don’t provide your bank or credit card details when applying for jobs. Don't transfer any money or complete suspicious online surveys. If you see something suspicious, report this job ad .