Lead Data Platform Engineer — Spark, dbt, Airflow (AWS)

ASX Operations

Sydney

On-site

AUD 180,000 - 260,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

ASX Group in Sydney seeks an experienced Data Engineering Leader to shape and run Australia's premier data platform. You will design end-to-end data solutions, lead data lake/mesh initiatives and mentor platform engineers.

The role requires 10+ years in data engineering, deep cloud expertise with AWS, Spark, dbt and Airflow, and the ability to partner with architecture, analytics and delivery teams to deliver strategic value.

Qualifications

  • 10+ years of data engineering experience.
  • Proven track record designing and operating large-scale data platforms.
  • Strong collaboration with architecture, analytics and product teams.

Responsibilities

  • Design end-to-end data platform solutions including ingestion, storage, transformation and serving layers.
  • Lead data lake and data mesh architecture design for scalability and cost efficiency.
  • Build reusable data pipelines and developer tooling to accelerate delivery.
  • Engage stakeholders across architecture, delivery and business SMEs to ensure optimal data solutions.
  • Ensure pipelines comply with engineering standards and governance.
  • Champion DataOps including CI/CD, IaC and automated testing.
  • Collaborate with security and data office to enforce privacy and regulatory controls.

Skills

Data engineering leadership
AWS cloud architecture
Distributed systems design
CI/CD for data pipelines

Tools

Apache Spark
dbt
Airflow
Terraform
AWS Redshift
Apache Iceberg

Job description

ASX Group in Sydney seeks an experienced Data Engineering Leader to shape and run Australia's premier data platform. You will design end-to-end data solutions, lead data lake/mesh initiatives and mentor platform engineers.

The role requires 10+ years in data engineering, deep cloud expertise with AWS, Spark, dbt and Airflow, and the ability to partner with architecture, analytics and delivery teams to deliver strategic value.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Platform Engineer - AWS, Data Lakehouse, DataOps
Lead Data Platform Engineer - AWS, Data Lakehouse, DataOps

ASX Limited • Australia

Hybrid
AUD 150,000 - 190,000
Hybrid working
Flexible working arrangements
Lead Data Platform Engineer: Databricks, Snowflake & AWS
Lead Data Platform Engineer: Databricks, Snowflake & AWS

Cox Purtell • Sydney

On-site
AUD 150,000 - 220,000
Principal Data Engineer (Apache Spark, dbt, Airflow)
Principal Data Engineer (Apache Spark, dbt, Airflow)

ASX Operations • Sydney

On-site
AUD 180,000 - 260,000
Data Engineer, AWS & Real-Time Platform (Hybrid Sydney)
Data Engineer, AWS & Real-Time Platform (Hybrid Sydney)

ASX Limited • Sydney

Hybrid
AUD 180,000 - 240,000
Hybrid working
Flexible working arrangements
Lead Data Engineer: AI & AWS Data Platforms
Lead Data Engineer: AI & AWS Data Platforms

Deloitte Services • Sydney

On-site
AUD 180,000 - 280,000
Retail discounts
Wellbeing leave
Volunteer days
+3
AWS Data Engineer - Real-Time Data Platform (Hybrid)
AWS Data Engineer - Real-Time Data Platform (Hybrid)

ASX Limited • Sydney

Hybrid
AUD 120,000 - 170,000
Hybrid work options
Senior Data Engineer
Senior Data Engineer

Cox Purtell • Sydney

On-site
AUD 150,000 - 220,000
Senior Data Engineer
Senior Data Engineer

Cox Purtell • Sydney

On-site
AUD 150,000 - 220,000
Lead Data Engineer – AI, AWS & Data Platforms
Lead Data Engineer – AI, AWS & Data Platforms

Deloitte Services Pty Ltd • Sydney

On-site
AUD 180,000 - 240,000
retail discounts
wellbeing leave
paid volunteering days
+3
Lead Data Engineer - Data & AI Platforms
Lead Data Engineer - Data & AI Platforms

Ippon Positive Technology • Melbourne

On-site
AUD 140,000 - 190,000
Market leading salaries
Work life balance
4 day a week option
+1
Top locations