Senior Data Engineer – Databricks / Aws

Otic Group Pty; Melbourne; Sur place; AUD 140 000 - 180 000; Plein temps

Résumé du poste

Otic Group Pty is seeking a Senior Data Engineer to design, build and deploy enterprise data platforms and pipelines in cloud environments from Melbourne. The role requires strong hands-on experience with Databricks, PySpark, Python, AWS and modern data architectures.

You will own ETL/ELT pipelines, data lake and data warehouse patterns, and contribute to CI/CD automation. The position is Melbourne-based with permanent residency or rights to work in Australia required.

Qualifications

• 7+ years of data engineering experience.
• Strong hands-on experience designing, building and deploying enterprise data platforms.
• Advanced Python and PySpark development skills.
• Strong AWS cloud experience for data engineering solutions.
• Experience building ETL/data pipelines with Apache Spark and dbt.
• Terraform / IaC experience, ideally provisioning Databricks.
• Experience implementing automated CI/CD pipelines (GitHub Actions).
• Understanding data lake, data warehouse and cloud data architectures.

Responsabilités

• Design and build scalable data architectures and pipelines for enterprise data platforms.
• Develop and maintain ETL/data pipelines using Spark and dbt in cloud environments.
• Implement infrastructure as code for Databricks environments with Terraform.
• Set up and maintain CI/CD pipelines for data engineering workflows.
• Collaborate with stakeholders to translate business requirements into practical data solutions.

Connaissances

Python

PySpark

AWS

Data architectures

Stakeholder communication

7+ years experience

CI/CD pipelines

Databricks

Apache Spark

dbt

Git

GitHub Actions

Terraform

Outils

Databricks

Apache Spark

dbt

Terraform

GitHub Actions

Git

Description du poste

“Otic” means smart people doing smart work, together.

We are a wholly owned Australian company committed to helping our clients design and build intelligent software solutions that unlock value in their business.

Otic Group was formed to provide talented technology professionals an opportunity to work with not only some of the most prominent companies in Australia, but work with incredibly talented peers in an exciting, dynamic young environment.

We are seeking a Senior Data Engineer with strong hands-on experience designing, building and deploying enterprise data platforms and pipelines within cloud environments.

Key skills and experience required:

• Strong commercial experience with Databricks, including development of scalable data architectures and data pipelines.
• Advanced Python and PySpark development skills.
• Strong AWS cloud experience supporting data engineering solutions.
• Experience building and maintaining ETL/data pipelines using technologies such as Apache Spark and dbt.
• Strong Terraform / Infrastructure as Code experience, ideally provisioning and managing Databricks environments.
• Experience implementing automated CI/CD pipelines, preferably using GitHub Actions or similar tooling.
• Strong understanding of modern data lake, data warehouse and cloud data architecture patterns.
• Experience processing and transforming large-scale datasets.
• Strong software engineering practices including Git, testing, deployment automation and code quality.
• Ability to translate business and technical requirements into practical, scalable data solutions.
• Strong stakeholder communication skills and the ability to operate independently in a complex enterprise environment.
• Ideally 7+ years of data engineering experience with recent hands-on delivery in large enterprise environments.

Over and above having these technical skills, you will also have a strong passion and flair for creating professional prototypes and dashboards.

Being a Melbourne based organisation, we’re looking for our team to be based locally. We also require applicants to hold an Australian Citizenship, Permanent Residency (PR), or have other permanent work rights.