Senior Data Engineer - Real-Time Pipelines on AWS & Airflow

ASX; Sydney; Hybrid; AUD 140,000 - 200,000; Full time

Benefits offered by this job

Hybrid working

Learning & development

Job summary

ASX, Australia's leading securities exchange, is seeking a senior data platform engineer to shape how data is ingested, curated and delivered across the business.

Embedded in the Data & Integration Platform within ASX Technology, you will collaborate with data owners, build scalable pipelines, and deliver files, reports, dashboards and APIs.

You'll apply DevSecOps practices, work with AWS, Airflow, Kafka and data warehousing concepts in a market infrastructure environment.

Qualifications

• Hands-on engineer with experience AWS ingestion, curation, storage and distribution.
• Knowledge of data modelling and data warehousing best practices.
• Experience with data mesh concepts and federated data ownership.

Responsibilities

• Collaborate with stakeholders to understand data objectives and champion the central data platform.
• Work with domain-aligned data product teams alongside engineering resources.
• Ingest core data assets and curate into source-aligned data products; create files, reports, dashboards, APIs.
• Contribute to data engineering practice and drive standards, frameworks and automation.
• Ensure pipelines and products run in a critical market infrastructure environment.

Skills

Python

Spark

DevSecOps

Data modelling

Data warehousing

Federated data ownership

Capital Markets knowledge

Data pipelines

Education

Bachelor's degree in Computer Science or related field

Tools

Airflow

AWS services

Confluent Kafka

AWS Glue

Apache Iceberg

Athena

RedShift

Tableau/Power BI/QuickSight

Job description

ASX, Australia's leading securities exchange, is seeking a senior data platform engineer to shape how data is ingested, curated and delivered across the business.

Embedded in the Data & Integration Platform within ASX Technology, you will collaborate with data owners, build scalable pipelines, and deliver files, reports, dashboards and APIs.

You'll apply DevSecOps practices, work with AWS, Airflow, Kafka and data warehousing concepts in a market infrastructure environment.