Data Engineer

Umbrella Club; Sydney; On-site; AUD 120,000 - 180,000; Full time

Job summary

Umbrella Club is seeking an experienced Data Engineer to join a team focused on building and supporting scalable, cloud-native data platforms in Sydney. You will design, develop and support data pipelines using AWS services and Spark/PySpark, with a strong emphasis on data modelling, governance and CI/CD practices.

The role requires hands-on SQL and Python, experience with CDC and operational databases, and collaboration with stakeholders in an Agile delivery environment.

Qualifications

• Design, develop and support cloud-native data platforms and pipelines.
• Hands-on experience with SQL and Python; strong AWS knowledge (S3, Glue, EMR, Athena, Aurora PostgreSQL).
• Experience with CDC, data governance, CI/CD and monitoring; Agile delivery environment.

Responsibilities

• Design, develop and support cloud-native data platforms and data pipelines.
• Build and maintain ETL/ELT pipelines using AWS services.
• Work with SQL, Python and Spark/PySpark for data processing.
• Develop with AWS S3, Glue, EMR, Athena and Aurora PostgreSQL.
• Apply data modelling and database design principles for scalable solutions.
• Work with CDC and operational databases.
• Support data governance, monitoring and CI/CD processes.
• Collaborate with stakeholders and delivery teams in Agile environments.
• Troubleshoot data pipeline and platform issues and contribute to improvements.

Skills

SQL

Python

Spark/PySpark

Agile delivery

Stakeholder engagement

Problem-solving

Tools

AWS S3

AWS Glue

AWS EMR

AWS Athena

Aurora PostgreSQL

GitHub Actions

Terraform

dbt

Snowflake

Job description

We are looking for an experienced Data Engineer to join a team focused on building and supporting scalable, cloud-native data platforms.

What you’ll be doing

• Design, develop and support cloud-native data platforms and data pipelines.
• Build and maintain ETL/ELT pipelines using AWS services.
• Work extensively with SQL, Python and Spark/PySpark for data processing and transformation.
• Develop solutions using AWS S3, Glue, EMR, Athena and Aurora PostgreSQL.
• Apply data modelling and database design principles to support scalable data solutions.
• Work with Change Data Capture (CDC) and operational databases.
• Contribute to data governance, monitoring and CI/CD processes.
• Work with GitHub Actions to support automated deployments and delivery.
• Collaborate with stakeholders and delivery teams in an Agile environment.
• Troubleshoot data pipeline and platform issues and contribute to continuous improvement.

What we’re looking for

• Strong commercial experience as a Data Engineer.
• Hands-on expertise in SQL and Python.
• Strong experience with AWS, particularly S3, Glue, EMR, Athena and Aurora PostgreSQL.
• Strong knowledge of data modelling.
• Hands-on experience with Spark/PySpark.
• Experience with CDC, data governance, CI/CD and monitoring.
• Experience working with operational databases.
• Strong problem-solving and stakeholder engagement skills.
• Experience working in Agile delivery environments.

Nice to have

• Kafka / Amazon MSK
• Terraform
• Event-driven architectures
• Snowflake
• dbt