Principal Data Engineer

M&T Resources; Sydney; On-site; AUD 150,000 - 190,000; Full time

Job summary

M&T Resources is seeking an experienced Data Architect to lead the design and development of scalable, secure data solutions on AWS in Sydney. This 50/50 hands-on role will architect and optimise data pipelines, mentor engineers, and collaborate with stakeholders to deliver high-performance data products.

You will also have the opportunity to implement SageMaker and influence data governance, security, and reliability across the platform.

Qualifications

• Strong AWS data platform and pipeline experience.
• Expertise in Python, Spark, dbt and Airflow.
• Strong knowledge of distributed systems and data architecture.
• Experience with Redshift, Iceberg and data lakehouse patterns.
• Proven experience leading data platform design and governance.

Responsibilities

• Lead end-to-end data platform architecture.
• Design scalable data lake and data mesh solutions.
• Build reusable data engineering frameworks and tools.
• Partner with stakeholders to deliver effective data solutions.
• Drive DataOps, CI/CD, automation and observability.
• Embed security, privacy, governance and reliability.

Skills

AWS data platform

Python

Spark

dbt

Airflow

data architecture

distributed systems

Redshift

Iceberg

data lakehouse

data governance

Tools

SageMaker

Job description

Are you ready to architect the future of data in a high-performance, mission-critical environment?

Come and join this established enterprise data team and lead and own the design and development of scalable, secure data solutions on AWS.

In this 50/50 hands on role you’ll architect and optimise data pipelines, mentor engineers, and collaborate with stakeholders to deliver high-performance, future-ready data products.

You’ll also have an opportunity to implementSageMaker!

Key responsibilities:

• Lead end-to-end data platform architecture
• Design scalable data lake and data mesh solutions
• Build reusable data engineering frameworks and tools
• Partner with stakeholders to deliver effective data solutions
• Drive DataOps, CI/CD, automation and observability
• Embed security, privacy, governance and reliability

What we’re looking for:

• Strong AWS data platform and pipeline experience.
• Expertise in Python, Spark, dbt and Airflow.
• Strong knowledge of distributed systems and data architecture.
• Experience with Redshift, Iceberg and data lakehouse patterns.
• Proven experience leading data platform design and governance.