Data Engineer ( GCP )

XPT Software; Sydney; On-site; AUD 120,000 - 180,000; Full time

Job summary

XPT Software in Sydney, Australia, is seeking a data engineer to design, build, and optimise batch and streaming data pipelines using approved frameworks and tooling. You will ensure pipelines are resilient, performant, and cost‑efficient while integrating monitoring, data quality checks, and governance‑friendly practices.

The role requires 2–4 years in data engineering, strong SQL, knowledge of cloud platforms, and experience with data orchestration tools.

Qualifications

• 2–4 years in a data engineering or closely related role.
• Proficient in SQL for data wrangling, transformation, and analysis.
• Experience building ETL/ELT pipelines in cloud or distributed environments.
• Familiarity with cloud platforms and services.
• Knowledge of data modeling concepts.
• Experience with data orchestration tools.
• Familiar with version control and CI/CD pipelines.
• Understanding of data quality, validation, and observability practices.
• Works independently with minimal supervision on well-scoped tasks.
• Collaborates effectively with stakeholders.
• Communicates clearly and documents work for others to maintain or build upon.
• Open to code reviews, feedback, and continuous learning.
• Able to troubleshoot and resolve data pipeline failures or performance issues.
• Has delivered production-ready pipelines end-to-end.

Responsibilities

• Design, build, and optimise batch and streaming data pipelines using approved frameworks and tooling.
• Ensure pipelines are resilient, performant, and cost optimised.
• Implement monitoring, alerting, and automated recovery processes.
• Apply data quality checks and reconciliation processes to ensure accuracy and completeness.
• Maintain metadata, lineage, and documentation to support governance requirements.
• Adhere to role-based access control, data privacy, and security standards.
• Contribute to the development and maintenance of ingestion frameworks, transformation layers, and data models.
• Implement and maintain CI/CD pipelines for automated testing and deployment.
• Optimise data storage and retrieval strategies to meet performance needs.
• Work closely with Technical Leads, Data Architects, Analysts, and business stakeholders to deliver fit-for-purpose solutions.
• Participate in code reviews and knowledge-sharing sessions to improve team capability.
• Explore and adopt new tools, frameworks, and techniques to enhance engineering capability.
• Identify opportunities to automate and streamline workflows.

Skills

SQL

ETL/ELT pipelines

Cloud platforms

Data modeling

Data orchestration tools

Version control

CI/CD pipelines

Data quality/observability

Independent work

Stakeholder collaboration

Code reviews

Troubleshooting data pipelines

Production-ready pipelines

Tools

Git

CI/CD tooling

Data orchestration tools

Cloud services

Job description

• Design, build, and optimise batch and streaming data pipelines using approved frameworks and tooling.
  
• Ensure pipelines are resilient, performant, and cost optimised.
  
• Implement monitoring, alerting, and automated recovery processes.
  
• Apply data quality checks and reconciliation processes to ensure accuracy and completeness.
  
• Maintain metadata, lineage, and documentation to support governance requirements.
  
• Adhere to role-based access control, data privacy, and security standards.
  
• Contribute to the development and maintenance of ingestion frameworks, transformation layers, and data models.
  
• Implement and maintain CI/CD pipelines for automated testing and deployment.
  
• Optimise data storage and retrieval strategies to meet performance needs.
  
• Work closely with Technical Leads, Data Architects, Analysts, and business stakeholders to deliver fit-for-purpose solutions.
  
• Participate in code reviews and knowledge-sharing sessions to improve team capability.
  
• Explore and adopt new tools, frameworks, and techniques to enhance engineering capability.
  
• Identify opportunities to automate and streamline workflows.
  

Key Skills & Experience

• Proficient in SQL for data wrangling, transformation, and analysis
  
• Experience building ETL/ELT pipelines in cloud or distributed environments
  
• Familiarity with cloud platforms and services
  
• Knowledge of data modeling concepts
  
• Experience with data orchestration tools
  
• Familiar with version control and CI/CD pipelines
  
• Understanding of data quality, validation, and observability practices
  
• Works independently with minimal supervision on well-scoped tasks
  
• Collaborates effectively with stakeholders
  
• Communicates clearly and documents work for others to maintain or build upon
  
• Open to code reviews, feedback, and continuous learning
  
• Able to troubleshoot and resolve data pipeline failures or performance issues
  
• 2–4 years in a data engineering or closely related role
  

Key Responsibilities

• Design, build, and optimise batch and streaming data pipelines using approved frameworks and tooling.
  
• Ensure pipelines are resilient, performant, and cost optimised.
  
• Implement monitoring, alerting, and automated recovery processes.
  
• Apply data quality checks and reconciliation processes to ensure accuracy and completeness.
  
• Maintain metadata, lineage, and documentation to support governance requirements.
  
• Adhere to role-based access control, data privacy, and security standards.
  
• Contribute to the development and maintenance of ingestion frameworks, transformation layers, and data models.
  
• Implement and maintain CI/CD pipelines for automated testing and deployment.
  
• Optimise data storage and retrieval strategies to meet performance needs.
  
• Work closely with Technical Leads, Data Architects, Analysts, and business stakeholders to deliver fit-for-purpose solutions.
  
• Participate in code reviews and knowledge-sharing sessions to improve team capability.
  
• Explore and adopt new tools, frameworks, and techniques to enhance engineering capability.
  
• Identify opportunities to automate and streamline workflows.
  

Key Skills & Experience

• Proficient in SQL for data wrangling, transformation, and analysis
  
• Strong programming skills
  
• Experience building ETL/ELT pipelines in cloud or distributed environments
  
• Familiarity with cloud platforms and services
  
• Knowledge of data modeling concepts
  
• Experience with data orchestration tools
  
• Familiar with version control and CI/CD pipelines
  
• Understanding of data quality, validation, and observability practices
  
• Works independently with minimal supervision on well-scoped tasks
  
• Collaborates effectively with stakeholders
  
• Communicates clearly and documents work for others to maintain or build upon
  
• Open to code reviews, feedback, and continuous learning
  
• Able to troubleshoot and resolve data pipeline failures or performance issues
  
• 2–4 years in a data engineering or closely related role
  
• Has delivered production-ready pipelines end-to-end
  

Demonstrated ability to improve code quality, reliability, or performance