Databricks - Practice Lead / Data & Analytics Consulting

Find My New Job; Melbourne; On-site; AUD 180,000 - 260,000; Full time

Job summary

Find My New Job is seeking an experienced Databricks Practice Lead to drive technical vision, growth, and delivery excellence for our Databricks practice across enterprise clients.

You will lead architecture discussions, mentor engineers, and coordinate with delivery managers to ensure scalable cloud data solutions while staying current with Databricks capabilities and AI-assisted development practices.

Qualifications

• 8+ years in data engineering or related fields.
• 3+ years leading technical teams or architecture initiatives.
• Deep Databricks expertise including Unity Catalog and Delta Lake.
• Strong Azure expertise with ADLS, ADF, Event Hubs, Key Vault, DevOps.
• Proficiency in Python with OOP principles.
• Experience with Airflow or Dagster for orchestration.
• Experience with streaming tech like Kafka or Event Hubs.
• Strong data architecture and enterprise data platform design.
• Experience implementing data governance, quality, security and compliance.
• Experience deploying cloud-native data solutions on Azure.
• Excellent consulting, presentation, and stakeholder management skills.
• Ability to lead multiple initiatives across distributed teams.
• Experience in Agile environments.

Responsibilities

• Lead the Databricks Practice technical direction and standards.
• Design enterprise-scale data platform architectures using Databricks.
• Provide technical leadership across client engagements.
• Partner with clients to translate business challenges into data solutions.
• Lead discovery sessions, architecture reviews, and design discussions.
• Mentor Data Engineers, Tech Leads, and Solution Architects.
• Define best practices for data engineering, DevOps, security, governance.
• Drive ETL/ELT development, data quality, monitoring, performance.
• Promote Lakehouse, Delta Lake, and event-driven architectures.
• Collaborate with Delivery Managers for project planning and risk management.
• Support pre-sales with design, proposals, estimates, and PoCs.
• Stay current with Databricks capabilities and emerging tech.
• Promote AI-assisted software development practices.

Skills

Leadership

Stakeholder management

Consulting

Presentation skills

Tools

Databricks

Azure Data Factory

Azure Data Lake Storage

Azure DevOps

Airflow

Dagster

Kafka

Azure Event Hubs

Python

Job description

Job Description

We are seeking an experienced Databricks Practice Lead to drive the technical vision, growth, and delivery excellence of our clients new Databricks practice.

This role combines deep hands-on expertise in modern data engineering with leadership responsibilities, including defining technical standards, mentoring engineering teams, supporting business development, and delivering scalable cloud-based data solutions for enterprise clients.

You will work closely with clients, engineering teams, solution architects, delivery leaders, and business stakeholders to design innovative data platforms, guide technical decision-making, and expand the organization's Databricks capabilities. The ideal candidate is both a strong technical leader and a trusted advisor who can bridge business objectives with modern data engineering solutions.

Responsibilities

• Lead the technical direction and growth of the Databricks Practice, establishing standards, best practices, and reusable solution accelerators.
• Design and review enterprise-scale data platform architectures using Databricks.
• Provide technical leadership and architectural guidance across multiple client engagements.
• Partner with clients to understand business challenges and recommend scalable data and analytics solutions.
• Lead technical discovery sessions, solution workshops, architecture reviews, and design discussions with clients.
• Mentor and coach Data Engineers, Technical Leads, and Solution Architects, fostering technical excellence and career development.
• Define best practices for data engineering, software engineering, DevOps, security, governance, and operational excellence.
• Establish standards for ETL/ELT development, data quality, monitoring, performance optimization, and production support.
• Drive adoption of modern data architectures, including Lakehouse, Delta Lake, and event-driven data processing.
• Collaborate with Delivery Managers and Engineering Managers to support project planning, staffing, technical risk management, and solution quality.
• Support pre-sales activities, including solution design, technical proposals, effort estimation, client presentations, and proof-of-concept initiatives.
• Stay current with Databricks platform capabilities and emerging technologies, identifying opportunities to improve services and expand practice offerings.
• Promote the adoption of AI-assisted software development and engineering best practices across the practice.

Qualifications

• 8+ years of professional experience in Data Engineering, Data Platform Engineering, or related disciplines.
• 3+ years of experience leading technical teams, architecture initiatives, or data engineering practices.
• Deep expertise with Databricks, including workspace administration, notebooks, jobs, workflows, Unity Catalog, Delta Lake, and Lakehouse architecture.
• Extensive experience with Microsoft Azure, including services such as Azure Data Lake Storage (ADLS), Azure Data Factory, Azure Event Hubs, Azure Key Vault, and Azure DevOps.
• Strong programming skills in Python, including clean coding practices and object-oriented programming principles.
• Experience with orchestration tools such as Apache Airflow or Dagster.
• Experience working with streaming technologies such as Apache Kafka or Azure Event Hubs.
• Strong understanding of data architecture, dimensional data modeling, metadata management, and enterprise data platform design.
• Experience implementing data governance, data quality, security, and compliance practices.
• Experience building and optimizing enterprise ETL/ELT solutions.
• Strong knowledge of SQL and relational database concepts.
• Experience deploying cloud-native data solutions, preferably on Microsoft Azure.
• Excellent consulting, presentation, and stakeholder management skills with experience interacting directly with enterprise clients.
• Experience leading architecture discussions, technical governance, and solution reviews.
• Ability to lead multiple initiatives simultaneously while providing technical direction across distributed teams.
• Experience working in Agile development environments.