Mid/Senior Data Engineer: Build Scalable Data Pipelines

XPT Software Australia; Sydney; On-site; AUD 90,000 - 130,000; Full time

Job summary

XPT Software Australia in Sydney is seeking a mid to senior data engineer to join its data team. The role focuses on building data pipelines, profiling and transformation on a modern platform, and ensuring data quality.

You will work with SQL and Python or Spark, automate reconciliation between legacy and target systems, provide masked test data for non-production environments, and document pipelines in partnership with ASIC data staff.

Qualifications

• Essential: data engineering on a modern platform.
• Proficient in SQL and Python or Spark.
• Experience with data quality and reconciliation.

Responsibilities

• Build version controlled pipelines for profiling, cleansing, transformation and migration.
• Automate reconciliation between legacy and target (counts, control totals, field comparisons).
• Provide masked test data for non production environments.
• Document pipelines and pair with ASIC data staff.

Skills

Data engineering

SQL

Python or Spark

Tools

Databricks

Mainframe data extraction

Informix

Job description

XPT Software Australia in Sydney is seeking a mid to senior data engineer to join its data team. The role focuses on building data pipelines, profiling and transformation on a modern platform, and ensuring data quality.

You will work with SQL and Python or Spark, automate reconciliation between legacy and target systems, provide masked test data for non-production environments, and document pipelines in partnership with ASIC data staff.