Principal Data Platform Engineer — Lead Real-Time Pipelines

Devopsroles; Sydney; Hybrid; AUD 180,000 - 280,000; Full time

Benefits offered by this job

Amazing Darlinghurst office space

Team lunches on Monday & Wednesdays

Training & Conference fund

Autonomy, influence and room to grow

Job summary

Simple Machines is a data and software engineering consultancy seeking a Principal Data and Platform Engineer to lead design and build of cloud-native data platforms. You’ll own architecture decisions, drive data mesh patterns, and partner with clients to deliver real outcomes.

You’ll design scalable data ecosystems on Databricks, Snowflake, AWS and GCP, implementing data contracts, products, and robust pipelines while mentoring engineers and shaping engineering standards.

Qualifications

• Strong Python and SQL experience.
• Deep experience with Spark and modern data platforms (Databricks / Snowflake).
• Solid grasp of cloud data services (AWS or GCP).
• Infrastructure-as-code experience (Terraform, Pulumi).
• CI/CD pipelines with GitHub Actions and ArgoCD.
• Data testing and quality frameworks (dbt, Great Expectations, Soda).

Responsibilities

• Lead Platform & Architecture Design for cloud-native data platforms.
• Build modern data platforms using data mesh concepts and data contracts.
• Develop high-performance batch and real-time data pipelines.
• Work at scale with large datasets and robust production systems.
• Consult with clients, shape solutions, and mentor teams.
• Set engineering standards and ensure observability and quality.

Skills

Python

SQL

Spark

Tools

Databricks

Snowflake

AWS

GCP

Terraform

Pulumi

GitHub Actions

ArgoCD

dbt

Great Expectations

Soda

Job description

Simple Machines is a data and software engineering consultancy seeking a Principal Data and Platform Engineer to lead design and build of cloud-native data platforms. You’ll own architecture decisions, drive data mesh patterns, and partner with clients to deliver real outcomes.

You’ll design scalable data ecosystems on Databricks, Snowflake, AWS and GCP, implementing data contracts, products, and robust pipelines while mentoring engineers and shaping engineering standards.