Cloud Data Engineer: AWS, PySpark & Lakehouse Migration

Profusion Group; City of Melbourne; On-site; AUD 120,000 - 160,000; Full time

Job summary

Profusion Group is seeking a Data Engineer for a 12-month contract with a major enterprise organisation. You will own data assets from ingestion through transformation, reconciliation, and production delivery, in a cloud lakehouse migration using AWS, PySpark, and Databricks.

Join a small, high-performing team in Melbourne and help modernise enterprise data platforms, delivering high-quality data solutions to critical business functions.

Qualifications

• Extensive data engineering experience in enterprise-scale environments.
• Advanced Python and SQL development skills.
• Hands-on Spark/PySpark and Databricks experience.
• Experience with AWS data services (Glue, Athena, S3, Lambda, Step Functions).
• Familiarity with CI/CD and IaC practices.

Responsibilities

• Design and build scalable AWS-based data pipelines with PySpark.
• Drive data quality, reconciliation, and platform reliability.
• Lead data transformations and production delivery in a lakehouse migration.
• Collaborate with business and tech stakeholders to deliver trusted data outcomes.
• Support Databricks migration initiatives.

Skills

Python

SQL

Spark / PySpark

AWS data services

Data modelling

Tools

Databricks

Delta Lake

CI/CD

Git

Infrastructure as Code

Job description

Profusion Group is seeking a Data Engineer for a 12-month contract with a major enterprise organisation. You will own data assets from ingestion through transformation, reconciliation, and production delivery, in a cloud lakehouse migration using AWS, PySpark, and Databricks.

Join a small, high-performing team in Melbourne and help modernise enterprise data platforms, delivering high-quality data solutions to critical business functions.