Senior Data Platform Architect - Spark dbt Airflow (Hybrid)

ASX; Sydney; Hybrid; AUD 180,000 - 240,000; Full time

Benefits offered by this job

Hybrid working

Flexible working arrangements

Job summary

ASX is seeking a senior Data Engineer to evolve and lead the end-to-end data platform across the organisation, including ingestion, storage, transformation and serving layers. You will design scalable data lake and data mesh architectures in a secure AWS environment, building reusable tooling to accelerate delivery across teams.

You will collaborate with architecture, product owners and workload engineers to ensure compliant, reliable data solutions and drive DataOps practices, including CI/CD,

Qualifications

• 10+ years of data engineering experience.
• Proven track record of designing and operating large-scale data platforms and pipelines in AWS cloud environment.
• Experience with enterprise data pipeline tooling (Apache Spark, dbt, Airflow).
• Deep expertise in distributed systems, batch and streaming data processing and modern data architecture patterns.

Responsibilities

• Evolve the end-to-end data platform solution design including ingestion, storage, transformation and serving layers across the organisation.
• Lead the design of data lake and data mesh architectures ensuring scalability, fault tolerance and cost efficiency.
• Design, build and maintain reusable frameworks, libraries and internal developer tooling to standardise and accelerate data pipeline development across teams.
• Identify, engage and influence stakeholders including architecture disciplines, product owners, delivery owners, workload engineers and business SMEs.
• Ensure data pipelines and engineering processes adhere to existing engineering standards and compliance reporting.

Skills

Data engineering

AWS

Spark

dbt

Airflow

Terraform

Data governance

Data lakes

ETL

Python

Tools

dbt

Airflow

Terraform

Redshift

Apache Iceberg

Job description

ASX is seeking a senior Data Engineer to evolve and lead the end-to-end data platform across the organisation, including ingestion, storage, transformation and serving layers. You will design scalable data lake and data mesh architectures in a secure AWS environment, building reusable tooling to accelerate delivery across teams.

You will collaborate with architecture, product owners and workload engineers to ensure compliant, reliable data solutions and drive DataOps practices, including CI/CD,