Senior Spark & Scala Data Engineer - Real-Time Pipelines

Peoplebank; Australia; Hybrid; AUD 111,000 - 185,000; Part time

Job summary

Peoplebank is seeking an experienced Spark & Scala Data Engineer to join a secure enterprise environment supporting critical infrastructure. Design and build scalable data pipelines using Spark, Scala, and the Hadoop ecosystem in an on-site setting.

This rolling contract role offers a day-per-fortnight onsite arrangement across Australia’s major cities. You will mentor engineers, implement CI/CD, and collaborate with stakeholders to deliver high-quality, timely data solutions.

Qualifications

• Extensive experience in Data Engineering or Big Data development.
• Hands-on expertise with Spark, Scala and Java.
• Experience with real-time and batch data pipelines.
• Strong SQL and data analysis skills.
• Familiarity with CI/CD, automated testing and Agile delivery.

Responsibilities

• Design and develop scalable data pipelines using Spark and Scala.
• Build and support batch and real-time data processing solutions.
• Analyse requirements and provide fit-for-purpose designs.
• Develop APIs, data services and reusable engineering frameworks.
• Deliver high-quality, well-tested code and perform code reviews.
• Support CI/CD, automation and production operations.
• Collaborate with stakeholders to deliver projects on time and within budget.
• Mentor junior engineers and contribute to continuous improvement.

Skills

Spark

Scala

Java

SQL

CI/CD

Agile delivery

Stakeholder management

Big Data

Tools

Kafka

Hive

HBase

HDFS

Hadoop ecosystem

Job description

Peoplebank is seeking an experienced Spark & Scala Data Engineer to join a secure enterprise environment supporting critical infrastructure. Design and build scalable data pipelines using Spark, Scala, and the Hadoop ecosystem in an on-site setting.

This rolling contract role offers a day-per-fortnight onsite arrangement across Australia’s major cities. You will mentor engineers, implement CI/CD, and collaborate with stakeholders to deliver high-quality, timely data solutions.