Technical Lead: Databricks Lakehouse on AWS

Cox Purtell Staffing Services; Sydney; On-site; AUD 180,000 - 240,000; Full time

Benefits offered by this job

Dedicated wellbeing leave

Parental leave

Job summary

Cox Purtell Staffing Services is seeking a highly skilled Technical Lead / Principal Engineer to design and own end-to-end Databricks Lakehouse platforms on AWS. You will build medallion architectures, batch and streaming pipelines, and drive cost and performance across large-scale data programs.

You may lead a team or work hands-on, coaching engineers while setting technical direction. The role emphasizes platform ownership over isolated components and access to vendor training.

Qualifications

• End-to-end data platform experience, infrastructure through to data lake.
• Hands-on Databricks and Delta Lake with medallion architecture and PySpark/Scala on AWS.
• Advanced SQL and performance tuning across distributed systems with Spark and cloud troubleshooting.
• Databricks certifications (Data Engineer / Professional) and AWS certs highly regarded.
• Technical Lead level: strong individual delivery on complex builds.

Responsibilities

• Architect end-to-end Databricks Lakehouse platforms on AWS (Glue or Redshift).
• Design and build batch, streaming, and real-time pipelines for AI and analytics workloads.
• Own distributed systems performance, tuning, and cost management at scale.
• Improve CI/CD and infrastructure as code (Terraform) and automated deployment.

Skills

Databricks expertise

Delta Lake

Apache Spark

PySpark

Scala

Medallion Architecture

AWS

Terraform

CI/CD

Data Engineering

Data Architecture

Data Platform

Tools

Databricks

Delta Lake

Glue

Redshift

Terraform

CI/CD pipelines

Job description

Cox Purtell Staffing Services is seeking a highly skilled Technical Lead / Principal Engineer to design and own end-to-end Databricks Lakehouse platforms on AWS. You will build medallion architectures, batch and streaming pipelines, and drive cost and performance across large-scale data programs.

You may lead a team or work hands-on, coaching engineers while setting technical direction. The role emphasizes platform ownership over isolated components and access to vendor training.