Senior Data Engineer: AWS, Spark, Hadoop & Airflow

Maltem Australia; Sydney; Hybrid; AUD 150,000 - 190,000; Full time

Benefits offered by this job

Flexible working practices

Ongoing training and development

Career progression discussions

Job summary

Maltem Australia is seeking a Senior Data Engineer for a Banking/Financial Services client. You’ll manage Big Data platforms on AWS (S3/EC2) with Spark, Hadoop (HDFS, Hive, Impala) and Presto, guided by Airflow.

This hybrid Sydney CBD role involves collaboration with multiple teams, continuous improvement, and a clear path for career progression, with remuneration offered for permanent or contractor arrangements.

Qualifications

• Experience with Hadoop ecosystem including HDFS, Hive and Impala.
• Experience with Spark and cloud enablement, setting up AWS services.
• Strong SQL scripting and Python programming experience.
• Airflow experience for job orchestration would be highly regarded.
• Growth mindset with problem solving and learning new technologies.
• Strong communication and stakeholder management across multiple teams.
• Ability to work in a hybrid fashion in Sydney CBD 2 days a week.
• Must have full working rights in Australia.

Responsibilities

• Run Spark jobs on AWS, using Airflow for orchestration and scheduling.
• Manage the AWS framework, governance and guardrails to enable growth.
• Collaborate with internal business/technical teams on cloud/data enablement solutions.
• Provide continuous improvement recommendations to stakeholders and platform owners.

Skills

AWS

Spark

Python

SQL

Airflow

Tools

Hadoop

HDFS

Hive

Impala

Presto

S3

EC2

Job description

Maltem Australia is seeking a Senior Data Engineer for a Banking/Financial Services client. You’ll manage Big Data platforms on AWS (S3/EC2) with Spark, Hadoop (HDFS, Hive, Impala) and Presto, guided by Airflow.

This hybrid Sydney CBD role involves collaboration with multiple teams, continuous improvement, and a clear path for career progression, with remuneration offered for permanent or contractor arrangements.