Senior Data Engineer: Databricks + AWS, Pipelines & CI/CD

Otic Group Pty; Melbourne; On-site; AUD 140,000 - 180,000; Full time

Job summary

Otic Group Pty is seeking a Senior Data Engineer to design, build and deploy enterprise data platforms and pipelines in cloud environments from Melbourne. The role requires strong hands-on experience with Databricks, PySpark, Python, AWS and modern data architectures.

You will own ETL/ELT pipelines, data lake and data warehouse patterns, and contribute to CI/CD automation. The position is Melbourne-based with permanent residency or rights to work in Australia required.

Qualifications

• 7+ years of data engineering experience.
• Strong hands-on experience designing, building and deploying enterprise data platforms.
• Advanced Python and PySpark development skills.
• Strong AWS cloud experience for data engineering solutions.
• Experience building ETL/data pipelines with Apache Spark and dbt.
• Terraform / IaC experience, ideally provisioning Databricks.
• Experience implementing automated CI/CD pipelines (GitHub Actions).
• Understanding data lake, data warehouse and cloud data architectures.

Responsibilities

• Design and build scalable data architectures and pipelines for enterprise data platforms.
• Develop and maintain ETL/data pipelines using Spark and dbt in cloud environments.
• Implement infrastructure as code for Databricks environments with Terraform.
• Set up and maintain CI/CD pipelines for data engineering workflows.
• Collaborate with stakeholders to translate business requirements into practical data solutions.

Skills

Python

PySpark

AWS

Data architectures

Stakeholder communication

7+ years experience

CI/CD pipelines

Databricks

Apache Spark

dbt

Git

GitHub Actions

Terraform

Tools

Databricks

Apache Spark

dbt

Terraform

GitHub Actions

Git

Job description

Otic Group Pty is seeking a Senior Data Engineer to design, build and deploy enterprise data platforms and pipelines in cloud environments from Melbourne. The role requires strong hands-on experience with Databricks, PySpark, Python, AWS and modern data architectures.

You will own ETL/ELT pipelines, data lake and data warehouse patterns, and contribute to CI/CD automation. The position is Melbourne-based with permanent residency or rights to work in Australia required.