AWS Data Engineer - Kafka, API & Real-Time Pipelines

XPT Software

Melbourne

On-site

AUD 110,000 - 150,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

XPT Software in Melbourne is seeking a data engineering professional to design, build, and optimize data pipelines spanning batch, streaming, and near-real-time processing. Strong knowledge of Kafka, API implementation, PostgreSQL and AWS is essential.

The role requires designing reusable data engineering frameworks, accelerators, and templates to standardize ingestion, transformation, orchestration, and data quality controls, with a focus on production-grade pipelines, error handling, logging,

Qualifications

  • Must have strong experience in building, enhancing, and maintaining data pipeline frameworks supporting batch, streaming, and near-real-time processing.
  • Must be able to design reusable data engineering frameworks, accelerators, and templates to standardize ingestion, transformation, orchestration, and data quality controls.
  • Must demonstrate proficiency in ETL/ELT development, SQL-based transformations, and distributed data processing.
  • Must implement production-grade pipelines with appropriate error handling, logging, monitoring, and performance optimization.
  • Strong understanding of SDLC process.
  • Should have strong understanding and troubleshooting skills

Responsibilities

  • Develop, enhance, and maintain data pipelines supporting batch, streaming, and near-real-time processing.
  • Design reusable data engineering frameworks, accelerators, and templates for standardized ingestion, transformation, orchestration, and data quality controls.
  • Implement production-grade ETL/ELT pipelines with robust error handling, logging, monitoring, and performance optimization.

Skills

Kafka
API implementation
PostgreSQL
AWS

Tools

AWS Certified

Job description

XPT Software in Melbourne is seeking a data engineering professional to design, build, and optimize data pipelines spanning batch, streaming, and near-real-time processing. Strong knowledge of Kafka, API implementation, PostgreSQL and AWS is essential.

The role requires designing reusable data engineering frameworks, accelerators, and templates to standardize ingestion, transformation, orchestration, and data quality controls, with a focus on production-grade pipelines, error handling, logging,

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AWS Data Engineer: Real-Time Data Pipelines
AWS Data Engineer: Real-Time Data Pipelines

Xpt Software Australia Pty Ltd • Melbourne

On-site
AUD 110,000 - 140,000
Data Engineering Lead – Real-Time Pipelines & AWS
Data Engineering Lead – Real-Time Pipelines & AWS

XPT Software Australia Pty Ltd • Melbourne

On-site
AUD 120,000 - 160,000
Data Pipeline Engineer - AWS, Kafka & ETL
Data Pipeline Engineer - AWS, Kafka & ETL

XPT Software Australia • Melbourne

On-site
AUD 90,000 - 130,000
Data Engineer Lead
Data Engineer Lead

XPT Software Australia Pty Ltd • Melbourne

On-site
AUD 120,000 - 160,000
Data Engineering Lead - Kafka, ETL & AWS
Data Engineering Lead - Kafka, ETL & AWS

Ampstek • Melbourne

On-site
AUD 110,000 - 160,000
Data Engineer - Aws
Data Engineer - Aws

Xpt Software Australia Pty Ltd • Melbourne

On-site
AUD 110,000 - 140,000
Data Platform Engineer: Real-Time Ingestion & Decisioning
Data Platform Engineer: Real-Time Ingestion & Decisioning

XPT Software Australia • Sydney

On-site
AUD 120,000 - 180,000
Hybrid Data Engineer – Real‑Time Pipelines & AWS, Sydney
Hybrid Data Engineer – Real‑Time Pipelines & AWS, Sydney

ASX Limited • Sydney

Hybrid
AUD 129,000 - 203,000
Data Engineer - GCP Pipelines & BigQuery
Data Engineer - GCP Pipelines & BigQuery

XPT Software Australia Pty Ltd • Ryde

On-site
AUD 110,000 - 150,000
Data Engineer AWS - Real-Time Pipelines (Hybrid)
Data Engineer AWS - Real-Time Pipelines (Hybrid)

Asx • Sydney

Hybrid
AUD 180,000 - 240,000
Hybrid working options
Top locations