NEW - Data Engineer / Developer - Databricks - Perm & Contract

Find My New Job; New Zealand; On-site; NZD 110,000 - 170,000; Full time

Job summary

Find My New Job is assisting a client in Auckland with a senior Data Engineer role focused on Databricks-based data architectures. You will collaborate with data leaders, lead engineering teams, and guide clients on best practices in data engineering and cloud data platforms.

The role requires extensive hands-on experience with Databricks, Delta Lake, and Unity Catalog, plus a track record of delivering large-scale data lakehouse solutions for complex business needs.

Qualifications

• 8+ years of experience in data engineering or architecture.
• Deep expertise in Big Data Platforms and Cloud Data Warehouses.
• Strong understanding of Databricks on public clouds (AWS, Azure, GCP).
• Experience with designing scalable data models for Lakehouse architectures.

Responsibilities

• Design scalable data architectures using Databricks and Delta Lake features.
• Lead and mentor engineering teams, driving best practices in data management.
• Engage with clients to understand business challenges and deliver Databricks-based solutions.
• Provide technical leadership in building Data Lakehouse components to meet client needs.

Skills

Databricks

Data architecture

Leadership

Cloud platforms

Tools

Delta Lake

Delta Live Tables

Unity Catalog

Job description

Our client in Auckland is a leading ANZ AI and Data Analytics Consulting Firm that empowers organisations to drive better outcomes through data and technology.

For 30 years they have partnered with some of the largest healthcare, life sciences, financial services, and government entities across ANZ.

We have an exciting opportunity for a highly skilled Data Engineer to join their innovative and dynamic team.

Who You’ll Work With

Our client is a fiercely human business and technology consulting company that leads with outcomes to bring more value, in all ways, always.

From strategy through delivery, they collaborate with clients to bring powerful customer experiences, innovative ways of working, and new products and services to life.

What You’ll Do

• Collaborate closely with data leaders to drive sales, recruiting, account management, consulting, and operational excellence across the practice
• Be part of their growing Databricks Center of Excellence and focus on delivering/leading on Databricks projects.
• Act as a Databricks SME, supporting business development efforts and guiding clients through best practices in architecture, data engineering, and scalable analytics
• Drive innovation and performance through a team-based approach that values output, ownership, and employee wellbeing
• Lead and manage project risk—including planning, budgeting, deliverables, and executive-level alignment
• Deepen company presence in the market by developing proposals, SOWs, and strategies to grow their footprint within existing accounts
• Provide technical and architectural guidance on Databricks, modern data platforms, and cloud-native solutions to both clients and team members
• Be a mentor and thought leader, regularly recommending emerging technologies and tools that align with client goals and future-state data strategies
• Bring a business-first lens to every conversation, driving impactful, tech-enabled outcomes that elevate our clients’ competitive edge.

What You’ll Bring

As an Engineer in the Databricks practice, you will leverage your extensive experience in architecture design and expertise in data engineering technologies to craft innovative solutions that meet clients' data needs. You are a leader in your field, ready to experiment and drive projects forward.

Key Responsibilities

• Design and implement scalable data architectures using Databricks, with hands-on experience in specific platform features such as Delta Lake, Uniform (Iceberg), Delta Live Tables, and Unity Catalog.
• Lead and mentor engineering teams, fostering a culture of learning and innovation, while driving best practices in data management and performance optimization.
• Engage with clients to understand their business challenges and deliver solutions that align with their goals, utilizing Databricks' capabilities to enhance outcomes.
• Demonstrate hands on technical leadership in designing and developing different components of a Data Lakehouse platform to meet client’s business needs.

Requirements

• 8+ years of experience in data engineering or architecture, including 4+ years of direct experience with Databricks and specific products like Delta Lake, Delta Live Tables, and Unity Catalog.
• Deep expertise in Big Data Platforms and Cloud Data Warehouses.
• Strong understanding of Databricks platform technical architecture on public cloud platforms like AWS, Azure, GCP etc. with experience in standing up scalable Databricks environments from scratch.
• Strong expertise in building industry standard, extensible data models for Silver and Gold layers of the Lakehouse solution following design standards like Star Schema, Data Vault etc.
• Advanced proficiency in Object-Oriented programming languages (like Java, Python, PySpark) and NoSQL Databases including performance tuning and optimization of complex data pipeline solutions.
• Experience with Container Management Systems and AI/ML platforms.
• Strong skills in streaming data ingestion and modern data workflows.
• Experience in delivering large scale migration, modernization initiative from legacy Data Warehouses to Databricks Lakehouse platform.
• Exposure to Databricks consumption estimates calculation considering different Data & AI pipeline workloads for multiple environments.
• Exposure to building Disaster Recovery (DR) solution strategy and implementation for a large Databricks platform is a big plus.

An ideal candidate will have Databricks Data Engineering Professional certification completed with multiple complex Databricks project delivery experience.