Databricks Senior Data Engineer - Lead Scalable Pipelines

Tardis Group; Western Australia; On-site; AUD 140,000 - 190,000; Full time

Benefits offered by this job

12-month contract

Job summary

Tardis Group in Australia is seeking an experienced Senior Data Engineer to join the team on a 12-month contract, ASAP start. This hands-on role combines deep technical expertise with mentoring and elevating a data engineering team.

You will design and maintain scalable ETL/ELT pipelines with Databricks and Spark, architect migrations, ingestion across SQL/Postgres, APIs and Kafka, and drive CI/CD, governance and data catalog initiatives while coaching others.

Qualifications

• Senior Data Engineer with extensive Databricks and Spark experience.
• Strong SQL and Python proficiency.
• Experience with Airflow, dbt or similar orchestration tech.
• Experience in data migration, API integration and Kafka streaming.
• Familiar with CI/CD, infrastructure automation and data cataloging tools.
• Proven ability to mentor and upskill team members.
• Excellent communication and ability to explain complex tech clearly.

Responsibilities

• Design, build and maintain scalable ETL/ELT pipelines and enterprise data platforms using Databricks & Apache Spark.
• Architect data migration, replication and transformation solutions.
• Build robust ingestion pipelines across SQL/Postgres, APIs and Kafka/streaming data.
• Work with complex data structures such as JSON and optimise them for analytics and downstream consumption.
• Implement orchestration, CI/CD, automated infrastructure and platform governance.
• Mentor and coach data engineers, promoting strong engineering practices, design patterns and code quality.
• Communicate the why behind technical decisions and help build deeper data engineering capability across the team.

Skills

Databricks

Apache Spark

SQL

Python

Airflow

dbt

Orchestration

Kafka

CI/CD

Infrastructure automation

Data catalog tools

Mentoring

Communication skills

Job description

Tardis Group in Australia is seeking an experienced Senior Data Engineer to join the team on a 12-month contract, ASAP start. This hands-on role combines deep technical expertise with mentoring and elevating a data engineering team.

You will design and maintain scalable ETL/ELT pipelines with Databricks and Spark, architect migrations, ingestion across SQL/Postgres, APIs and Kafka, and drive CI/CD, governance and data catalog initiatives while coaching others.