Data Engineer - (Databricks)

Talent Corp; City of Brisbane; On-site; AUD 218,000 - 232,000; Full time

Job summary

Talent Corp is seeking an experienced Data Engineer in Brisbane to join a cloud and AI-focused data environment. You will design and implement Databricks-based data pipelines and Lakehouse solutions, leveraging AWS, Python, SQL and PySpark to deliver scalable data integrations.

The role requires hands-on Databricks expertise, production experience, and strong collaboration with engineering and analytics teams to improve data quality, governance and performance in a dynamic setting.

Qualifications

• 5+ years' Data Engineering experience.
• Databricks is mandatory and production-grade Databricks pipelines are required.
• Strong understanding of the Databricks Lakehouse Platform.
• Strong Spark/PySpark, Python and SQL skills.
• AWS experience across S3, Glue, Lambda, Athena, Redshift, IAM and API Gateway.
• Experience with cloud-native data integration and orchestration.
• Knowledge of APIs and modern integration patterns.
• Excellent communication and problem-solving abilities.

Responsibilities

• Build and optimise Databricks data pipelines and Lakehouse solutions.
• Develop scalable ingestion, transformation and orchestration workflows.
• Design and deliver cloud-native data integrations across AWS.
• Work extensively with Python, SQL, Spark/PySpark.
• Improve data performance, quality, security and governance.
• Support AI-enabled data and integration initiatives.
• Collaborate with engineering, architecture and business teams.
• Contribute to engineering standards, automation and best practices.

Skills

Databricks

AWS

Python

SQL

PySpark

Spark

Delta Lake

Unity Catalog

Medallion Architecture

Terraform/CloudFormation

CI/CD

Docker/Kubernetes

APIs & integration

Tools

Terraform/CloudFormation

CI/CD tooling

Docker/Kubernetes

Delta Lake

Unity Catalog

Medallion Architecture

Job description

We're looking for an experienced Data Engineer with strong, hands-on Databricks experience to join a modern cloud and AI-focused data environment.

Databricks is a mandatory requirement for this role. We're particularly interested in engineers who have spent several years building and optimising production data solutions on the Databricks Lakehouse Platform, rather than candidates with limited or incidental exposure.

The Role

You'll work across modern data engineering and cloud integration initiatives, designing scalable data solutions using Databricks, AWS, Python, SQL and PySpark.

What You'll Be Doing

• Build and optimise Databricks data pipelines and Lakehouse solutions
• Develop scalable ingestion, transformation and orchestration workflows
• Design and deliver cloud-native data integrations across AWS
• Work extensively with Python, SQL, Spark/PySpark
• Improve data performance, quality, security and governance
• Support AI-enabled data and integration initiatives
• Collaborate with engineering, architecture and business teams
• Contribute to engineering standards, automation and best practices

What We're Looking For

• 5+ years' Data Engineering experience
• Proven experience building and optimising production Databricks pipelines
• Strong understanding of the Databricks Lakehouse Platform
• Strong Spark/PySpark, Python and SQL
• Strong AWS experience across services such as S3, Glue, Lambda, Athena, Redshift, IAM and API Gateway
• Experience with cloud-native data integration and orchestration
• Understanding of APIs and modern integration patterns
• Strong communication and problem-solving skills

Experience with Delta Lake, Unity Catalog, Medallion Architecture, Terraform/CloudFormation, CI/CD, Docker/Kubernetes or AI development tools would be highly regarded. This is a 6 month contract with strong possibility for an extension paying $950 - $1000 per day inc super.