Mid/Senior Data Engineer — Pipelines & Data Quality

XPT Software; Melbourne; On-site; AUD 100,000 - 140,000; Full time

Job summary

XPT Software in Melbourne, Australia is looking for a mid to senior data engineer to join our data team. You will build and maintain version-controlled data pipelines for profiling, cleansing, transformation and migration, using SQL and Python or Spark on a modern platform.

You will also automate reconciliation between legacy and target systems, provide masked test data for non-production environments, and document pipelines while collaborating with ASIC data staff.

Qualifications

• Essential data engineering on a modern platform; SQL and Python or Spark; data quality and reconciliation.
• Desirable: Databricks; mainframe or Informix data extraction; registry or financial data.

Responsibilities

• Build version controlled pipelines for profiling, cleansing, transformation and migration.
• Automate reconciliation between legacy and target (counts, control totals, field comparisons).
• Provide masked test data for non production environments.
• Document pipelines and pair with ASIC data staff.

Skills

Data engineering

SQL

Python

Spark

Data quality

Reconciliation

Tools

Databricks

Mainframe

Informix

Job description

XPT Software in Melbourne, Australia is looking for a mid to senior data engineer to join our data team. You will build and maintain version-controlled data pipelines for profiling, cleansing, transformation and migration, using SQL and Python or Spark on a modern platform.

You will also automate reconciliation between legacy and target systems, provide masked test data for non-production environments, and document pipelines while collaborating with ASIC data staff.