Data Engineer: Build AI-Ready Data Pipelines & Platforms

TechnologyOne; City of Brisbane; On-site; AUD 120,000 - 180,000; Full time

Benefits offered by this job

Competitive remuneration package

Employee share plan

Salary packaging options

Purchase Leave program

Flexible hours

Free gym membership

Volunteer leave

Free breakfast and coffee

LinkedIn Learning access

Job summary

TechnologyOne is seeking a Data Engineer to design, build and maintain scalable data pipelines and models across the enterprise. You’ll replace manual integrations with secure, reusable data infrastructure, enabling trusted analytics and AI-ready foundations.

You will work across ERP, CRM and core systems, translating business requirements into practical solutions leveraging Databricks on AWS, Python, Spark and SQL, with Terraform and CI/CD in play.

Qualifications

• Experience with Databricks on a cloud data platform.
• Proven experience architecting data platform solutions using Databricks, AWS and custom systems.
• Strong engineering capability across Python, Spark and SQL.
• Experience designing Databricks Unity Catalog.
• Hands-on experience with Terraform, CI/CD, Git/Bitbucket workflows.
• Ability to translate business requirements into practical technical solutions.

Responsibilities

• Design and build reliable data pipelines and models for operational and analytical needs.
• Shape the data platform by building capabilities on the Databricks Platform as a Service.
• Design scalable data ingestion patterns aligned to security and access-control standards.
• Identify and remediate duplicate, incomplete, conflicting or poorly governed data across core enterprise systems.
• Establish common definitions for key enterprise measures to improve reporting, analytics and AI outputs.
• Build secure, highly performant and cost-effective cloud data solutions with monitoring and management controls.
• Design, implement and optimise Databricks batch and streaming workflows using Python, Spark and SQL.
• Continually monitor, troubleshoot and improve data pipelines, platform performance, reliability and cost efficiency.
• Proven experience architecting data platform solutions using Databricks, AWS and custom systems.
• Strong engineering capability across Python, Spark and SQL, including ETL best practices on Databricks.
• Experience designing, managing and implementing Databricks Unity Catalog.
• Experience building or remediating complex enterprise data estates, including data quality, common definitions and governance patterns.
• Hands-on experience designing and managing Terraform modules, CI/CD pipelines and Git or Bitbucket workflows.
• Experience designing scalable performance-monitoring solutions and event pipelines that are secure, performant and cost effective.
• The ability to translate business requirements into practical technical solutions; experience with Azure, AWS, Data Cloud, QuickSight or similar platforms is highly regarded.

Skills

Python

Spark

SQL

Databricks

Data pipelines

Data modelling

AWS

Terraform

CI/CD

Git/Bitbucket

Tools

Databricks

Terraform

Git

Bitbucket

Job description

TechnologyOne is seeking a Data Engineer to design, build and maintain scalable data pipelines and models across the enterprise. You’ll replace manual integrations with secure, reusable data infrastructure, enabling trusted analytics and AI-ready foundations.

You will work across ERP, CRM and core systems, translating business requirements into practical solutions leveraging Databricks on AWS, Python, Spark and SQL, with Terraform and CI/CD in play.