Senior Data Engineer

Peoplebank; Melbourne; On-site; AUD 110,000 - 165,000; Part time

Job summary

Peoplebank is seeking an experienced Spark & Scala Data Engineer to join an enterprise environment supporting critical infrastructure. The role covers multiple Australian cities and is onsite 1 day per fortnight with rolling 3-month contracts.

You will design scalable data pipelines with Spark and Scala, build real-time and batch processing solutions, and mentor junior engineers while collaborating with stakeholders to deliver projects on time and within budget.

Qualifications

• Extensive experience in Data Engineering or Big Data development.
• Hands-on Scala, Java and Apache Spark.
• Experience building real-time and batch data pipelines.
• Strong knowledge of Kafka, Hive, HBase, HDFS and the Hadoop ecosystem.
• Strong SQL and data analysis skills.
• Experience with CI/CD, automated testing and Agile delivery.
• Strong communication and stakeholder management skills.

Responsibilities

• Design and develop scalable data pipelines using Spark and Scala.
• Build and support batch and real-time data processing solutions.
• Analyse requirements and provide fit-for-purpose technical designs.
• Develop APIs, data services and reusable engineering frameworks.
• Deliver high-quality, well-tested code and perform code reviews.
• Support CI/CD, automation and production operations.
• Collaborate with stakeholders to deliver projects on time and within budget.
• Mentor junior engineers and contribute to continuous improvement.

Skills

Data Engineering

Scala

Java

Apache Spark

Kafka

Hive

HBase

HDFS

SQL

CI/CD & Agile

Stakeholder management

Tools

Kafka

Hive

HBase

HDFS

CI/CD tools

Job description

This role opportunity exists for an experiencedSpark & Scala Data Engineer to join a secure enterprise environment supporting critical infrastructure.
Location:Melbourne, Adelaide, Brisbane, Sydney, Canberra
Work Type:1x day per fortnight onsite
Tenure:rolling 3x month contracts

Key Responsibilities:

• Design and develop scalable data pipelines using Spark and Scala.
• Build and support batch and real-time data processing solutions.
• Analyse requirements and provide fit-for-purpose technical designs.
• Develop APIs, data services and reusable engineering frameworks.
• Deliver high-quality, well-tested code and perform code reviews.
• Support CI/CD, automation and production operations.
• Collaborate with stakeholders to deliver projects on time and within budget.
• Mentor junior engineers and contribute to continuous improvement.

Key Skills:

• Extensive experience in Data Engineering or Big Data development.
• Strong hands-on experience with Scala, Java and Apache Spark.
• Experience building real-time and batch data pipelines.
• Strong knowledge of Kafka, Hive, HBase, HDFS and the Hadoop ecosystem.
• Experience designing and supporting large-scale Big Data platforms.
• Strong SQL and data analysis skills.
• Experience with CI/CD, automated testing and Agile delivery.
• Strong communication and stakeholder management skills.

Peoplebank and Leaders IT are committed to creating a diverse and inclusive workplace where everyone belongs. We welcome applications from people of all backgrounds, identities, and experiences. If you need adjustments to the recruitment process due to your circumstances, please let us know—we’re here to support you.