Lead Data&AI Engineer - Databricks

Mantel; Melbourne; Hybrid; AUD 150,000 - 230,000; Full time

Benefits offered by this job

New phone

Laptop

Swag

Flexible hybrid work

Job summary

Mantel is an Australian-owned IT Professional Services consultancy specializing in data and AI. We’re seeking a Lead Data & AI Engineer - Databricks to design and deploy modern data architectures, lead data pipelines and promote best practices across client projects.

You will mentor a multi-cloud data team, define data strategies for AI, and collaborate with stakeholders to deliver high-quality, scalable data solutions in a hybrid work environment.

Qualifications

• Strong consulting experience with external clients or internal teams.
• Commercial data experience across industries and domains
• Ability to engage with senior level stakeholders and influence teams
• Experience establishing technical architecture and leading delivery of technical projects and teams
• Prior experience leading greenfield projects and maintaining high-quality work throughout all project phases
• Technical brilliance across cloud-native services
• Expertise in data engineering, Lakehouse architectures and Spark-based processing
• Proficiency in SQL and Python (PySpark) or Scala; deep Spark engine knowledge
• Hands-on Databricks experience and modern cloud ecosystems (AWS/GCP/Azure)
• Knowledge of data security and data quality frameworks
• Ability to explain technical concepts to any audience
• Alignment with Mantel principles and professional judgment

Responsibilities

• Design and deploy data projects and data pipelines.
• Build data models and advanced queries for clients.
• Design greenfield solutions and improve existing ones.
• Lead data architectures on Databricks across clouds.
• Define data strategy for AI, governance, quality and lineage.
• Balance thorough, best-practice implementations with pragmatic solutions.
• Understand SOWs and commercial success of projects.
• Mentor team members and promote up-skilling.
• Educate clients about Mantel’s Databricks offerings and assist in pre-sales.
• Build trusted client relationships and influence stakeholders.
• Lead teams by Mantel’s principles and drive impact.

Skills

Consulting experience

Stakeholder engagement

Data engineering concepts

Cloud data architectures

SQL

Python (PySpark)

Scala

Tools

Databricks

Unity Catalog

Delta Live Tables

Spark

AWS

Azure

GCP

Job description

Mantel is an Australian-owned IT Professional Services consultancy, focused on emerging technology to help advance our clients by leveraging AI, Digital, Cloud, Data & Security technology. We are a fully AI-enabled technology consultancy and the partner that bridges the gap between blue-sky ambition and practical steps to deliver value. Since our inception in November 2017, we have experienced remarkable growth across Australia & New Zealand and are honoured to be recognised as a Great Place to Work for 6 years in a row!

We hire smart and talented people and get out of their way. As a principle-based organisation, we have a flat structure with no hierarchy. By focusing on our five principles and not getting caught up in red tape, we trust you to get the job done!

Data

Working in data at Mantel means you'll be ahead of the curve. You can align to a particular cloud platform or remain cloud agnostic, specialising in data engineering, machine learning, analytics, or data science. You'll design and scale enterprise-level, fully automated Agentic AI solutions, and speed up data platform migrations.

About the role

As a key member of our Data Engineering team, you’ll be leading design and deployment of data projects, developing data pipelines, building data models and advanced queries to help our clients harness the power of data.

As a Lead Data&AI Engineer - Databricks you will:

• Design and implement greenfield solutions, as well as recommending and implementing changes to existing solutions.
• Build and deploy modern data architectures (Medallion/Lakehouse) primarily on Databricks, utilizing cloud platforms like AWS, Azure, or GCP.
• Define and establish technical architectures and delivery frameworks, ensuring a clear, shared vision across the team and stakeholders.
• Have responsibility for the deliverables and efforts of project teams (varying in size, complexity and makeup of the team), and at times sharing responsibility collaboratively with other Leads / Principals.
• Define the data strategy for AI within the data domain, setting the standards for feature governance, data quality, and data lineage required for robust ML systems.
• Balance thorough, best-practice implementations with pragmatic solutions, depending on project timelines.
• Be deeply familiar with the SoW driving the work, understanding how to achieve commercial success of your project.
• Ensure quality of work is maintained through mentoring and up-skilling of others.
• Show outstanding consulting, technical and project management skills. Keep your skills sharpened, staying at the forefront of technologies relevant to your capability through continuous development, including gaining certifications. Share your learnings back with the team.
• Educate clients and potential clients about Mantel’s Databricks offerings relative to the overall strategy. Participate in pre-sales activities e.g. creating proposals/SoWs and facilitating walk-throughs with clients.
• Build meaningful and trusted relationships with clients, key decision makers and partners. Establish your reputation and influence as a data expert within Mantel and the broader community.
• Lead teams by demonstrating Mantel’s principles and attributes.

We want to hear from you if you have:

• Strong consulting experience, either with external clients or internally within an organisation
• Commercial data experience across different industries and domains
• Ability to engage with senior level stakeholders and having a broad influence in the internal and client’s teams
• Experience in establishing technical architecture and leading the delivery of technical projects and project teams
• Prior experience leading greenfield projects and maintaining high-quality work throughout all project phases
• Technical brilliance across a broad range of cloud-native services
• Expertise in a broad range of data engineering approaches, Lakehouse architecture and Spark-based processing.
• Proficiency in data languages: SQL and Python (PySpark) or Scala; having a deep understanding of the Spark engine is a significant plus.
• Demonstrable leading end to end data engineering projects, including using automation best practices for CI/CD and IaC.
• Hands‑on expertise in Databricks (Workflows, Unity Catalog, Delta Live Tables) and modern cloud ecosystems (AWS/GCP/Azure).
• Knowledge of data security best practices and data quality frameworks, ensuring data integrity and consistency
• The ability to explain technical concepts regardless of the audience
• At Mantel, we want you to be yourself. We trust our team members to use their professional judgment while treating colleagues and clients with respect, compassion, and professionalism in alignment with our five principles.
• You take ownership, always stepping up to help and prioritising your teammates. You're also a clear, kind communicator who makes sound decisions based on good judgement.

We value diverse skill-sets, and we'd love to hear from you even if you don't meet all of the above. If you feel you cover the majority of the experiences/skills listed we still encourage you to apply or get in touch.

What you can expect from us:

• We believe in unique experiences for all. Our My Deal program allows you to tailor your yearly plan, with the support of your Leader, to decide on what’s most important to you. That might be extra professional development, extra annual leave, time to work on your side hustle, or something else completely different! One size does not fit all.
• You’ll get all the tools you need to hit the ground running including a new phone, laptop & swag.
• A true investment in your AI-first career. We are a fully AI-enabled consultancy and provide you with the tools to lead. This includes access to the Mantel AI Gateway, our secure, centralised platform with multiple AI models, plus a dedicated monthly budget and a safe environment to experiment and innovate.
• You’ll be genuinely supported by an organisation that cares about not only you but your family as well, Mantel offers Flexible Personal Leave options for those unplanned moments in life.
• We support a flexible hybrid approach to working which is guided by our principles; we trust each other to “make good choices” about the best workplace locations for the requirements of the project, role and client. This can change based on our client needs.
• We know you won’t have one job for life. At Mantel, we believe in supporting our team to take their career in a direction that aligns with their passions. We have internal opportunities across Cloud, Data, Digital, Delivery & Security.
• Our elite partnerships are a direct result of our deep platform mastery. For you, this means you'll be working alongside teams with the highest levels of technical expertise and tackling the most complex, high-profile projects on the market. Check out our partner awards and recognitions here.
• Mantel is a certified Family Inclusive Workplace, which means we are committed to improving your wellbeing at work through leading practices on flexible work, parental leave, family care and family wellbeing