Data Engineer-Unified data platform

XPT Software; Sydney; On-site; AUD 110,000 - 150,000; Full time

Job summary

XPT Software is seeking an experienced Data Engineer to ingest and transform data from multiple source domains into the platform, following design principles and governance. You will build batch and streaming pipelines and orchestrate workflows across services.

Collaborate with DevOps, Data Architects, and operations teams to ensure end-to-end reliability and observability. You will work with real-time decisioning platforms such as Pega to support personalization use cases and data availability

Qualifications

• Experience with cloud-native data engineering patterns and object storage-based data offload architectures.
• Familiarity with real-time decisioning platforms (e.g. Pega) integration with data engineering layers.
• Exposure to MarTech or Transformation data programs in large enterprises.
• Experience with workflow orchestration tools such as Apache Airflow.
• Understanding of data governance, data lineage, and data quality processes.
• Aware of Agile or SAFe delivery methodologies and cross-functional squads.

Responsibilities

• Ingest and transform data from CRM feeds, third-party data, and digital signals into the platform according to governance standards.
• Build and maintain batch and streaming pipelines and workflow orchestration.
• Document data dictionary and cloud storage design for data offload scenarios.
• Collaborate with DevOps, Data Architects, and operations teams for end-to-end reliability and observability.
• Ensure accurate and timely data availability for personalization and customer engagement use cases.
• Participate in cross-team planning, technical reviews, and dependency management for decisioning workflows.
• Deliver across multiple use cases in an Agile squad model; manage dependencies across platforms and teams.
• Author and maintain technical documentation, including solution design notes and deployment runbooks.
• Provide SIT, UAT, and data validation support across environments with QA and operations teams.

Job description

Ingestion and transformation of data from multiple source domains (e.g. CRM feeds, third-party data, digital signals) into the platform following approved design principles and data governance standards.

Building and maintaining batch and streaming transform pipelines and workflow orchestration.

Data dictionary documentation and cloud object storage design for data offload scenarios.

Collaboration with extended team like DevOps, Data Architects, and operations teams to ensure end-to-end pipeline reliability and observability.

Demonstrate a sound understanding of how real-time decisioning platforms (such as Pega) consume and depend on data engineering outputs.

Work with decisioning and analytics teams to ensure accurate and timely data availability for personalisation and customer engagement use cases.

Participate in cross-team planning and technical reviews as they relate to data dependencies for decisioning workflows.

Cross-Platform & Delivery

Deliver across multiple use cases simultaneously in an Agile squad model, managing dependencies across platforms and teams.

Author and maintain technical documentation, including solution design notes, implementation plans, and deployment runbooks.

Actively participate in sprint ceremonies, backlog refinement, and cross-team dependency management.

Provide SIT, UAT, and data validation support across environments; work with QA and operations teams to validate data integrity pre- and post-release.

Skills & Experience

Familiarity with cloud-native data engineering patterns and object storage-based data offload architectures (e.g. AWS S3, Azure Data Lake).

Understanding of real-time decisioning platforms such as Pega, and how they integrate with data engineering layers.

Exposure to MarTech or Transformation data programs in a large enterprise or telco context.

Experience with workflow orchestration tools such as Apache Airflow or equivalent.

Understanding of data governance frameworks, data lineage, and data quality processes.

Knowledge of Agile or SAFe delivery methodology and working in cross-functional squads

Be careful - Don’t provide your bank or credit card details when applying for jobs. Don't transfer any money or complete suspicious online surveys. If you see something suspicious, report this job ad .