Senior Real-Time Data Engineer - Databricks & AWS Streaming

Nuage; Melbourne; On-site; AUD 90,000 - 130,000; Full time

Job summary

Nuage, a leading Australian brand, is building the streaming side of a major platform program in Melbourne. We are seeking a data engineer who can ensure real-time data behaves under changing platform conditions.

You will build and run streaming pipelines on Databricks and AWS, from ingestion through to trusted analytics-ready data, with a focus on latency, reliability, and cost efficiency.

Qualifications

• Strong hands-on Databricks streaming experience in production.
• Solid Python coding experience.

Responsibilities

• Build and own streaming pipelines using Databricks Structured Streaming, Auto Loader and Delta Lake.
• Ingest from Kafka (MSK) and Kinesis into a Bronze, Silver, Gold lakehouse.
• Keep data correct: dedup, late and out-of-order events, schema changes and safe replays.
• Monitor and tune pipelines for latency, cost and reliability.

Skills

Databricks streaming

Python

Kafka / Kinesis

AWS data services

CI/CD / IaC

Tools

Delta Lake

MSK

S3

Glue

Lambda

Athena

EMR

CloudWatch

Job description

Nuage, a leading Australian brand, is building the streaming side of a major platform program in Melbourne. We are seeking a data engineer who can ensure real-time data behaves under changing platform conditions.

You will build and run streaming pipelines on Databricks and AWS, from ingestion through to trusted analytics-ready data, with a focus on latency, reliability, and cost efficiency.