Senior Data Engineer: Reconciliation & Pipelines

Xpt Software Australia Pty Ltd; Canberra; On-site; AUD 120,000 - 180,000; Full time

Job summary

Xpt Software Australia Pty Ltd in Canberra is seeking a mid to senior Data Engineer or Analyst to design and operate data pipelines for profiling, cleansing, transformation and migration. You will automate reconciliation between legacy and target systems, provide masked test data for non production environments, and document pipelines while collaborating with ASIC data staff.

Essential skills include SQL and Python or Spark, data quality and reconciliation; Databricks and mainframe data

Qualifications

• Essential: data engineering on a modern platform; SQL and Python or Spark; data quality and reconciliation.
• Desirable: Databricks; mainframe or Informix data extraction; registry or financial data.

Responsibilities

• Build version controlled pipelines for profiling, cleansing, transformation and migration.
• Automate reconciliation between legacy and target (counts, control totals, field comparisons).
• Provide masked test data for non production environments.
• Document pipelines and pair with ASIC data staff.

Skills

SQL

Python

Spark

Data quality

Data reconciliation

Tools

Databricks

Job description

Xpt Software Australia Pty Ltd in Canberra is seeking a mid to senior Data Engineer or Analyst to design and operate data pipelines for profiling, cleansing, transformation and migration. You will automate reconciliation between legacy and target systems, provide masked test data for non production environments, and document pipelines while collaborating with ASIC data staff.

Essential skills include SQL and Python or Spark, data quality and reconciliation; Databricks and mainframe data