Senior Data Engineer: Spark on AWS and Cloud Modernisation

Xpert Consulting T/A Novon; Australia; On-site; AUD 90,000 - 150,000; Full time

Job summary

Xpert Consulting T/A Novon is seeking a seasoned Data Engineer to design, build, and optimise large-scale data pipelines using Spark and PySpark in an enterprise environment.

You'll work with AWS-native data solutions including Glue, Glue Catalog, S3, Airflow, DBT, and Starburst/Presto, migrating legacy assets to modern cloud architectures, ensuring data quality, performance, and scalable delivery while collaborating closely with architects and stakeholders.

Qualifications

• Minimum 5 years as a Data Engineer in enterprise-scale environments.
• Hands-on Spark and PySpark experience.
• Strong AWS data ecosystem knowledge (Glue, S3, Starburst/Presto, Airflow, DBT).

Responsibilities

• Design, build, and optimise data pipelines using Spark / PySpark.
• Develop and maintain AWS-native data solutions (Glue, Glue Catalog, S3, Airflow, DBT).
• Support migration of legacy pipelines to modern cloud architectures.
• Implement data quality, monitoring, and performance optimization.
• Collaborate with architects, analysts, and stakeholders to deliver scalable data solutions.

Skills

Spark

PySpark

AWS Cloud

Data Pipelines

Stakeholder Communication

Tools

AWS Glue

Glue Catalog

Amazon S3

Airflow

DBT

Starburst

Presto

Job description

Xpert Consulting T/A Novon is seeking a seasoned Data Engineer to design, build, and optimise large-scale data pipelines using Spark and PySpark in an enterprise environment.

You'll work with AWS-native data solutions including Glue, Glue Catalog, S3, Airflow, DBT, and Starburst/Presto, migrating legacy assets to modern cloud architectures, ensuring data quality, performance, and scalable delivery while collaborating closely with architects and stakeholders.