Principal Data Platform Engineer: Real-Time & Cloud-Native

Simple Machines; Sydney; On-site; AUD 180,000 - 260,000; Full time

Benefits offered by this job

Darlinghurst office

Team lunches

Training & conference fund

Job summary

Simple Machines is seeking a hands-on Principal Data and Platform Engineer to lead platform design and architectural decisions for modern, cloud-native data platforms. You will build greenfield data pipelines, work with data mesh concepts, and collaborate closely with clients to turn data into actionable outcomes.

You will design and implement scalable data ecosystems using Databricks, Snowflake, AWS and GCP, and you will guide teams on best practices, testing, and observability while

Qualifications

• Strong Python and SQL development experience in data platforms.
• Expertise with Spark and Databricks/Snowflake.
• Experience with AWS or GCP data services.
• Infrastructure-as-code using Terraform or Pulumi.
• CI/CD pipelines with GitHub Actions and ArgoCD.
• Data quality and testing frameworks like dbt/Great Expectations/Soda.

Responsibilities

• Lead Platform & Architecture Design: own end-to-end architecture of modern, cloud-native data platforms.
• Build Modern Data Platforms: design and deliver cloud-native data platforms using Databricks, Snowflake, AWS and GCP.
• Develop High-Performance Pipelines: build and optimise batch and real-time pipelines; work with streaming tech.
• Work at Scale: process large datasets with Spark and Flink; production-ready systems.
• Consult and Influence: work with clients to understand problems and shape pragmatic engineering solutions.
• Technical Leadership & Quality: set standards, review designs, raise data quality, observability and operational excellence.

Skills

Python

SQL

Spark

Tools

Terraform

Pulumi

GitHub Actions

ArgoCD

dbt

Great Expectations

Soda

Airflow

Kafka

Job description

Simple Machines is seeking a hands-on Principal Data and Platform Engineer to lead platform design and architectural decisions for modern, cloud-native data platforms. You will build greenfield data pipelines, work with data mesh concepts, and collaborate closely with clients to turn data into actionable outcomes.

You will design and implement scalable data ecosystems using Databricks, Snowflake, AWS and GCP, and you will guide teams on best practices, testing, and observability while