Senior Lead Data Engineer

Itbility; City of Melbourne; On-site; AUD 180,000 - 240,000; Full time

Job summary

Itbility in Melbourne, VIC seeks a Senior Lead Data Engineer to drive strategy, architecture, delivery and optimization of enterprise-scale data platforms on Microsoft Azure. You will lead data platform roadmaps, govern pipelines, data models, and lakehouse solutions while mentoring data engineering teams.

Strong leadership and stakeholder management are essential. Candidates should have 15+ years IT experience with 15+ years in Data Engineering, plus 10+ years leading large programs.

Qualifications

• 15+ years IT experience with 15+ years in Data Engineering.
• At least 10 years leading large-scale Data Engineering programs and teams.
• Proven record in cross-functional leadership and executive communication.

Responsibilities

• Lead end-to-end delivery of enterprise data platforms including architecture and deployment.
• Define and drive the data platform roadmap aligned with business priorities.
• Govern pipelines, models, warehouses, and lakehouse solutions on Azure.
• Provide technical leadership through design/code reviews and best practices.
• Champion Microsoft Fabric adoption, APIs, Azure Functions, and Python-based solutions.
• Ensure scalability, security, reliability, and cost optimization.
• Enforce data governance, quality, security, and compliance.
• Implement CI/CD, DevOps, and Agile delivery across data engineering.
• Collaborate with stakeholders to translate requirements into scalable tech.
• Mentor Data Engineering teams and foster continuous learning.
• Leverage AI tools to enhance productivity and solution quality.
• Drive innovation and platform modernization initiatives.

Skills

SQL

Python

PySpark

Azure Data Factory

Azure Synapse

Delta Lake

Data Modeling

Data Warehousing

Microsoft Fabric

Cloud-native Architecture

CI/CD

DevOps

Cost Optimization

Performance Tuning

Governance

Leadership

Stakeholder Mgmt

Tools

GitHub Copilot

Job description

Our client in Melbourne is looking for Senior Lead Data Engineer this is a contract 6+ months or Permanent role.

We are seeking a highly experienced Senior Lead Data Engineer to drive the strategy, architecture, delivery, and optimization of enterprise-scale data platforms on Microsoft Azure. The ideal candidate will combine deep technical expertise with strong leadership, stakeholder management, and delivery governance capabilities to build scalable, secure, and cost-efficient data solutions.

Must-Have Skills

• Strong expertise in SQL, Python, and PySpark.
• Extensive experience with Azure Data Factory (ADF) and Azure Synapse Analytics.
• Hands-on experience with Delta Lake architecture and modern data lakehouse implementations.
• Advanced knowledge of Data Modeling and Data Warehousing concepts.
• Strong exposure to the Microsoft Fabric ecosystem.
• Proven expertise in designing cloud-native data architectures and enterprise-scale solutions.
• Strong design thinking and problem-solving capabilities.
• Working knowledge of CI/CD pipelines and DevOps practices.
• Experience in cloud cost optimization, performance tuning, and resource governance.

Key Responsibilities

• Lead and own the end-to-end delivery of enterprise data platforms, including architecture, development, deployment, and optimization.
• Define and drive the data platform roadmap, ensuring alignment with business objectives and strategic priorities.
• Architect and govern scalable data pipelines, data models, data warehouses, and lakehouse solutions using Azure services.
• Provide technical leadership to Data Engineering teams through design reviews, code reviews, and implementation of engineering best practices.
• Drive the adoption and implementation of Microsoft Fabric, APIs, Azure Functions, and Python-based solutions.
• Ensure platform scalability, reliability, security, and operational excellence while optimizing cloud costs.
• Establish and enforce data governance, data quality, security, and compliance standards.
• Implement and drive CI/CD, DevOps, and Agile delivery practices across the data engineering landscape.
• Collaborate with business stakeholders, architects, and technology teams to translate business requirements into scalable technical solutions.
• Mentor, coach, and develop Data Engineering talent, fostering a culture of continuous learning and innovation.
• Leverage GitHub Copilot and AI-assisted engineering tools to improve productivity and solution quality.
• Drive innovation by evaluating emerging technologies and identifying opportunities for platform modernization.

Experience

• 15+ years of overall IT experience, including 15+ years in Data Engineering.
• Minimum 10+ years of leadership experience managing large-scale Data Engineering programs and teams.
• Proven track record of leading cross-functional teams and delivering complex enterprise data transformation initiatives.
• Experience building, hiring, and scaling high-performing Data Engineering teams.
• Experience working within multi-vendor and globally distributed delivery environments.
• Strong experience in stakeholder management, delivery governance, and executive-level communication.

Preferred Skills

• Experience building and managing enterprise APIs and Azure Function Apps.
• Exposure to framework-based data engineering development approaches.
• Experience in solution architecture, stakeholder engagement, and delivery governance.
• Azure Solutions Architect Expert
• Familiarity with MLOps, Data Science, and AI-driven analytics solutions.
• Hands-on experience with Power BI and enterprise reporting platforms.
• Experience leveraging AI and Generative AI capabilities within Data Engineering ecosystems.
• Strong ownership mindset with a focus on accountability and business outcomes.
• Excellent communication, stakeholder management, and influencing skills.
• Ability to balance technical leadership with delivery and people management responsibilities.
• Strategic thinker with a strong focus on execution excellence.
• Strong conflict resolution and cross-functional collaboration skills.
• Adaptability and resilience in fast-paced, evolving environments.
• Passion for mentoring teams and fostering a culture of innovation and continuous improvement.

All candidates should have full working rights in Australia.

Only shortlisted candidates will be contacted for this role.

Senior Lead Data Engineer • Melbourne, VIC, AU