Technology Lead

XPT Software; City of Melbourne; On-site; AUD 120,000 - 180,000; Full time

Job summary

XPT Software in Melbourne is seeking a skilled Data Engineer to design, build and maintain data pipelines across batch, streaming and near-real-time systems. You will work closely with analytics teams to implement scalable ETL/ELT processes using GCP services like BigQuery, Dataflow, Dataproc, Pub/Sub and Cloud Storage.

The role requires strong SQL proficiency, solid SDLC understanding, and hands-on troubleshooting abilities, with a focus on producing robust, auditable pipelines for enterprise

Qualifications

• Must have strong experience in building, enhancing, and maintaining data pipeline frameworks for batch, streaming, and near-real-time processing.
• Design reusable data engineering frameworks, accelerators, and templates to standardize ingestion, transformation, orchestration, and data quality controls.
• Hands-on experience with GCP data services: BigQuery, Cloud Dataflow, Dataproc, Pub/Sub, Cloud Storage.
• Proficient in ETL/ELT development, SQL transformations, and distributed data processing.
• Production-grade pipelines with error handling, logging, monitoring, and performance optimization.

Responsibilities

• Good communication skills with knowledge on SQL, GCP and Kafka.
• Strong understanding of SDLC process.
• Should have strong understanding and trouble shooting skills.

Tools

BigQuery

Cloud Dataflow

Dataproc

Pub/Sub

Cloud Storage

Job description

Must Have Skills:

GCP development Skills with Strong SQL knowledge

Nice to Have Skills:

• GCP Certified
• Kafka

Detailed Job Description:

• Good communication skills with knowledge on SQL, GCP and Kafka
• Strong understanding of SDLC process.
• Should have Strong understanding and trouble shooting skills

Core requirements:

• Must have strong experience in building, enhancing, and maintaining data pipeline frameworks supporting batch, streaming, and near-real-time processing.
• Must be able to design reusable data engineering frameworks, accelerators, and templates to standardize ingestion, transformation, orchestration, and data quality controls.
• Must have hands-on experience with GCP data services such as BigQuery, Cloud Dataflow, Dataproc, Pub/Sub, and Cloud Storage.
• Must demonstrate proficiency in ETL/ELT development, SQL-based transformations, and distributed data processing.
• Must implement production-grade pipelines with appropriate error handling, logging, monitoring, and performance optimization.