Data Engineer-Unified data platform

XPT Software Australia; Sydney; Vor Ort; AUD 120.000 - 180.000; Vollzeit

Zusammenfassung

XPT Software Australia is hiring a data engineer to design and optimise data ingestion and transformation pipelines, integrating data across multiple domains into cloud storage with governance in mind.

You will work on batch and streaming pipelines, collaborate with DevOps and analytics teams, and contribute to end-to-end data reliability and observability across decisioning workflows.

Qualifikationen

• Experience building data pipelines across multiple source systems.
• Familiarity with cloud storage and data offload architectures.
• Understanding of data governance, lineage and quality processes.
• Experience with real-time decisioning platforms (e.g., Pega) is a plus.

Aufgaben

• Ingest and transform data from multiple domains into the platform per governance standards.
• Build and maintain batch and streaming transform pipelines and workflow orchestration.
• Collaborate with DevOps, Data Architects, and operations teams to ensure end-to-end pipeline reliability and observability.
• Demonstrate data availability for personalization and customer engagement use cases across platforms.
• Participate in cross-team planning and technical reviews related to data dependencies for decisioning workflows.

Kenntnisse

Cloud-native data engineering

Pega integration

Data governance

Streaming & batch pipelines

Workflow orchestration

Agile SAFe

MarTech data programs

Data quality

Tools

Apache Airflow

Jobbeschreibung

Demonstrate a Sound Understanding Of

• Ingestion and transformation of data from multiple source domains (e.g. CRM feeds, third-party data, digital signals) into the platform following approved design principles and data governance standards.
• Building and maintaining batch and streaming transform pipelines and workflow orchestration.
• Data dictionary documentation and cloud object storage design for data offload scenarios.
• Collaboration with extended team like DevOps, Data Architects, and operations teams to ensure end-to-end pipeline reliability and observability.
• Decisioning Platform (Pega or equivalent)
• Demonstrate a sound understanding of how real-time decisioning platforms (such as Pega) consume and depend on data engineering outputs.
• Work with decisioning and analytics teams to ensure accurate and timely data availability for personalisation and customer engagement use cases.
• Participate in cross-team planning and technical reviews as they relate to data dependencies for decisioning workflows.
• Cross-Platform & Delivery
• Deliver across multiple use cases simultaneously in an Agile squad model, managing dependencies across platforms and teams.
• Author and maintain technical documentation, including solution design notes, implementation plans, and deployment runbooks.
• Actively participate in sprint ceremonies, backlog refinement, and cross-team dependency management.
• Provide SIT, UAT, and data validation support across environments; work with QA and operations teams to validate data integrity pre- and post-release.

Skills & Experience

• Familiarity with cloud-native data engineering patterns and object storage-based data offload architectures (e.g. AWS S3, Azure Data Lake).
• Understanding of real-time decisioning platforms such as Pega, and how they integrate with data engineering layers.
• Exposure to MarTech or Transformation data programs in a large enterprise or telco context.
• Experience with workflow orchestration tools such as Apache Airflow or equivalent.
• Understanding of data governance frameworks, data lineage, and data quality processes.
• Knowledge of Agile or SAFe delivery methodology and working in cross-functional squads