Senior Data Engineer / Consultant - Databricks

ICM Asia Pacific; City of Melbourne; On-site; AUD 140,000 - 180,000; Full time

Job summary

Adaptiv in Melbourne or Brisbane seeks a senior Data Engineer to design, build and deliver modern data platforms and engineering solutions with a focus on Databricks and cloud‑based architectures.

You’ll work across client engagements, collaborating with architects, consultants and engineering teams to translate business requirements into scalable, production‑ready data solutions. The role blends hands‑on coding with mentoring, in a growing Australian Data Practice.

Qualifications

• 6+ years' professional experience across Data Engineering, Data Platforms or a closely related discipline.
• Strong hands‑on experience delivering production Data Engineering solutions using Databricks.
• Practical experience with Databricks notebooks, workflows, Lakehouse architectures and modern data pipeline development.
• Strong Python, PySpark and SQL capability.
• Experience designing and developing ETL/ELT pipelines and complex data transformations.
• A good understanding of modern data architecture patterns, including Medallion architecture and scalable Lakehouse design.
• Strong data modelling knowledge, including dimensional, Star and Snowflake modelling approaches.
• Experience with cloud data platforms, ideally within Microsoft Azure or another major cloud ecosystem.
• Experience applying DevOps and software engineering practices including version control, CI/CD and automated deployment.
• Experience implementing testing, data quality and validation practices within production data environments.
• Familiarity with analytics and data consumption technologies such as Power BI or similar platforms.

Responsibilities

• Designing, building and optimising modern data pipelines across data lakes, lakehouses and data warehouses.
• Delivering production‑grade solutions using Databricks and associated cloud data technologies.
• Developing data transformations and engineering solutions using Python, PySpark and SQL.
• Working with Databricks notebooks, workflows and Lakeflow Declarative Pipelines (formerly Delta Live Tables).
• Applying modern data architecture patterns including Lakehouse and Medallion architectures.
• Designing and implementing robust data models for analytics, reporting and downstream data consumption.
• Applying software engineering and DevOps practices including Git, CI/CD, automated deployment and environment management.
• Implementing testing, data validation, monitoring and performance optimisation to ensure reliable production data solutions.
• Working directly with clients to understand requirements and translate business needs into pragmatic technical solutions.
• Leading technical discussions and communicating engineering concepts to both technical and non-technical stakeholders.
• Collaborating with architects and other consultants on solution design, estimation, scoping and delivery planning.
• Identifying delivery risks, communicating progress and helping resolve technical blockers.
• Conducting peer reviews and supporting high standards across code, documentation and engineering practices.
• Mentoring junior and intermediate engineers and contributing to knowledge sharing across the Data Practice.
• Helping improve Adaptiv's reusable frameworks, delivery patterns, templates and engineering standards.
• Staying current with developments across Databricks, cloud data platforms, AI/ML and the broader modern data ecosystem.

Skills

Databricks

Python

PySpark

SQL

ETL/ELT

Lakehouse

Medallion architecture

Cloud data platforms

DevOps practices

Power BI

client‑facing delivery

Tools

Azure

Databricks notebooks

Lakeflow

Job description

About Us

Adaptiv is a Kiwi success story with a growing presence across Australia and New Zealand. We specialise in integration, data and analytics, AI consulting and cloud solutions, helping organisations modernise their technology environments and unlock greater value from their data.

Our teams work with leading technologies including Databricks, Microsoft Azure, Microsoft Fabric, MuleSoft, Boomi and Solace to solve complex business and technology challenges.

What sets Adaptiv apart is our combination of deep technical capability and genuine consulting expertise. We work closely with our clients to understand their challenges, design pragmatic solutions and deliver technology that creates meaningful business outcomes.

As our Australian Data Practice continues to grow, we're looking for an experienced Senior Data Engineer / Data Consultant to join our team in Melbourne or Brisbane.

About the Role

This is a senior, hands‑on consulting role for someone who combines strong Data Engineering capability with the confidence to work directly with clients.

You’ll help design, build and deliver modern data platforms and engineering solutions, with a particular focus on Databricks and cloud‑based data architectures.

You’ll work across client engagements, collaborating with architects, consultants and engineering teams to translate business requirements into scalable, production‑ready data solutions.

This isn’t a role where you’ll simply provide technical oversight. You’ll remain close to the technology — designing solutions, building pipelines, writing code, solving complex engineering problems and helping ensure the quality of what we deliver.

You’ll also play an important role in supporting and mentoring other engineers as our Data Practice continues to grow across Australia.

What You’ll Be Doing

• Designing, building and optimising modern data pipelines across data lakes, lakehouses and data warehouses.


• Delivering production‑grade solutions using Databricks and associated cloud data technologies.


• Developing data transformations and engineering solutions using Python, PySpark and SQL.


• Working with Databricks notebooks, workflows and Lakeflow Declarative Pipelines (formerly Delta Live Tables).


• Applying modern data architecture patterns including Lakehouse and Medallion architectures.


• Designing and implementing robust data models for analytics, reporting and downstream data consumption.


• Applying software engineering and DevOps practices including Git, CI/CD, automated deployment and environment management.


• Implementing testing, data validation, monitoring and performance optimisation to ensure reliable production data solutions.


• Working directly with clients to understand requirements and translate business needs into pragmatic technical solutions.


• Leading technical discussions and communicating engineering concepts to both technical and non-technical stakeholders.


• Collaborating with architects and other consultants on solution design, estimation, scoping and delivery planning.


• Identifying delivery risks, communicating progress and helping resolve technical blockers.


• Conducting peer reviews and supporting high standards across code, documentation and engineering practices.


• Mentoring junior and intermediate engineers and contributing to knowledge sharing across the Data Practice.


• Helping improve Adaptiv's reusable frameworks, delivery patterns, templates and engineering standards.


• Staying current with developments across Databricks, cloud data platforms, AI/ML and the broader modern data ecosystem.

What You’ll Bring

• 6+ years' professional experience across Data Engineering, Data Platforms or a closely related discipline.


• Strong hands‑on experience delivering production Data Engineering solutions using Databricks.


• Practical experience with Databricks notebooks, workflows, Lakehouse architectures and modern data pipeline development.


• Strong Python, PySpark and SQL capability.


• Experience designing and developing ETL/ELT pipelines and complex data transformations.


• A good understanding of modern data architecture patterns, including Medallion architecture and scalable Lakehouse design.


• Strong data modelling knowledge, including dimensional, Star and Snowflake modelling approaches.


• Experience with cloud data platforms, ideally within Microsoft Azure or another major cloud ecosystem.


• Experience applying DevOps and software engineering practices including version control, CI/CD and automated deployment.


• Experience implementing testing, data quality and validation practices within production data environments.


• Familiarity with analytics and data consumption technologies such as Power BI or similar platforms.


• Strong communication skills and the confidence to work directly with clients and senior stakeholders.


• Experience working within consulting, professional services or client‑facing delivery environments, or demonstrable experience operating in a similar capacity.


• The ability to work autonomously while collaborating effectively across multidisciplinary delivery teams.

Exposure to Microsoft Fabric and the wider Microsoft Data ecosystem would be advantageous but isn't essential.

Relevant Databricks, Azure or other cloud certifications are highly regarded, along with a demonstrated commitment to ongoing professional development.

The Person We're Looking For

Technical capability matters, but we're looking for more than someone who can build a pipeline.

You’ll enjoy working with customers, asking the right questions and understanding the problem behind the technical requirement.

You’ll be comfortable taking ownership of your work, making pragmatic engineering decisions and explaining those decisions clearly to others.

You’ll enjoy helping other engineers develop and won't be afraid to challenge an approach when you believe there's a better way.

Most importantly, you’ll want to remain hands‑on while continuing to grow as a consultant and senior technical professional.

Why Adaptiv?

You’ll join a growing Data Practice where you’ll have the opportunity to influence how we deliver, not simply inherit established processes.

You’ll work with modern technologies, interesting clients and experienced people across Data, Integration, Cloud and AI.

We offer an environment where technical curiosity is encouraged, good ideas are heard and people are trusted to take ownership.

If you're an experienced Data Engineer looking for a role that combines hands‑on Databricks engineering, client engagement and genuine consulting responsibility, we'd love to hear from you.