Data Platform Engineer — Real-Time AWS & Spark Pro

M&T Resources; Sydney; On-site; AUD 110,000 - 150,000; Full time

Job summary

M&T Resources is seeking a Data Engineer to join a high-performing Data & Integration Platform team within a leading financial services organisation in Australia. This hands-on role focuses on modern, enterprise-scale data platforms, building scalable data products and pipelines across a complex tech environment.

You'll work with AWS technologies including Glue, Airflow, Iceberg, Athena, Redshift and DataZone to ingest, curate, store and distribute data, delivering analytics, reporting and APIs.

Qualifications

• Approximately 5 years of data engineering experience.
• Proficient in Python and Spark.
• Experience with Kafka and real-time streaming.
• Strong communication and stakeholder management skills.

Responsibilities

• Build and evolve enterprise data pipelines and products.
• Develop solutions across ingestion, curation, storage and distribution.
• Partner with data owners, SMEs and technology teams.
• Build real-time streaming solutions using Kafka and AWS.
• Develop reusable data frameworks and standards.

Skills

Data engineering

Python

Spark

Kafka

AWS

Airflow

Glue

Iceberg

Athena

Redshift

Communication

Stakeholder management

Tools

AWS

Airflow

Glue

Iceberg

Athena

Redshift

Job description

M&T Resources is seeking a Data Engineer to join a high-performing Data & Integration Platform team within a leading financial services organisation in Australia. This hands-on role focuses on modern, enterprise-scale data platforms, building scalable data products and pipelines across a complex tech environment.

You'll work with AWS technologies including Glue, Airflow, Iceberg, Athena, Redshift and DataZone to ingest, curate, store and distribute data, delivering analytics, reporting and APIs.