Senior Data Platform Engineer – Real-Time & Batch (Brisbane)

Task Recruitment; City of Brisbane; On-site; AUD 150,000 - 185,000; Full time

Job summary

Task Recruitment is hiring two Senior Software Engineers in Brisbane to build scalable data platforms handling high‑frequency telemetry from mining equipment. You’ll join a specialist team within a large global program spanning Australia, the US, Canada and India.

You will design and evolve backend systems using Java/Spring Boot, Python/Scala, and modern data tooling such as Spark, Delta Lake, and Kubernetes.

Qualifications

• Strong backend software engineering experience in Java/Spring Boot with ability to work across Python/Scala
• Experience designing microservices and distributed applications
• Experience with large data volumes and both streaming and batch processing
• Familiarity with data lake/lakehouse architectures (Delta Lake) and messaging/streaming tech (MQTT, Kafka)
• Experience with Docker and Kubernetes
• Collaboration across geographically distributed teams

Responsibilities

• Design scalable components and microservices for ingesting, processing and persisting high‑frequency telemetry data
• Build solutions for real‑time streaming and large‑scale batch processing
• Collaborate with data scientists to host and optimise Python/Scala/Java workloads
• Modernise Java/Spring Boot monolith toward a microservices architecture
• Containerise and deploy using Docker and Kubernetes
• Integrate with S3‑compatible object storage
• Contribute to architectural decisions around event sourcing, CQRS and hybrid cloud deployments
• Evaluate emerging tech to improve performance and scalability
• Work with teams across Australia, the US, Canada and India

Skills

Backend engineering

Java/Spring Boot

Python

Scala

Spark

Kubernetes

Delta Lake

Redis/Valkey

MQTT

PostgreSQL

Docker

Tools

Apache Spark

Delta Lake

Docker

Kubernetes

Redis/Valkey

MQTT

PostgreSQL

Kafka

Job description

Task Recruitment is hiring two Senior Software Engineers in Brisbane to build scalable data platforms handling high‑frequency telemetry from mining equipment. You’ll join a specialist team within a large global program spanning Australia, the US, Canada and India.

You will design and evolve backend systems using Java/Spring Boot, Python/Scala, and modern data tooling such as Spark, Delta Lake, and Kubernetes.