Azure Data Engineer

CareCone Group; City of Melbourne; On-site; AUD 180,000 - 240,000; Full time

Job summary

CareCone Group in Melbourne seeks an accomplished Lead Data Engineer to drive strategy, architecture, delivery and optimization of enterprise data platforms on Microsoft Azure. You will lead data teams, shape the roadmap, and ensure scalable, secure, cost-efficient solutions.

The role requires 10+ years IT/10+ years data engineering, strong leadership, stakeholder management, and hands-on expertise with Azure Data Factory, Synapse, Delta Lake, and modern data lakehouse architectures.

Qualifications

• 10+ years of IT experience including 10+ years in Data Engineering.
• Minimum 5+ years of leadership experience managing large-scale Data Engineering programs and teams.
• Experience working in multi-vendor and globally distributed delivery environments.

Responsibilities

• Lead and own the end-to-end delivery of enterprise data platforms, including architecture, development, deployment, and optimization.
• Define and drive the data platform roadmap, aligning with business objectives.
• Architect and govern scalable data pipelines, data models, data warehouses, and lakehouse solutions using Azure services.
• Provide technical leadership through design reviews, code reviews, and best-practices implementation.
• Drive adoption of Microsoft Fabric, APIs, Azure Functions, and Python-based solutions.
• Ensure scalability, reliability, security, and operational excellence while optimizing cloud costs.
• Establish and enforce data governance, data quality, security, and compliance standards.
• Implement CI/CD, DevOps, and Agile delivery practices across the data engineering landscape.
• Collaborate with stakeholders to translate business requirements into scalable technical solutions.
• Mentor and develop Data Engineering talent and foster continuous learning and innovation.
• Leverage AI-assisted tools to improve productivity and solution quality.
• Explore emerging technologies for platform modernization.

Skills

SQL

Python

PySpark

Azure Data Factory

Azure Synapse

Delta Lake

Data Modeling

Data Warehousing

Microsoft Fabric

Cloud-native Data Architecture

Design Thinking

CI/CD

DevOps

Cost Optimization

Performance Tuning

Resource Governance

Tools

Power BI

Azure Functions

Job description

We are seeking a highly experienced Lead Data Engineer to drive the strategy, architecture, delivery, and optimization of enterprise-scale data platforms on Microsoft Azure. The ideal candidate will combine deep technical expertise with strong leadership, stakeholder management, and delivery governance capabilities to build scalable, secure, and cost-efficient data solutions.

Must-Have Skills

• Strong expertise in SQL, Python, and PySpark
• Extensive experience with Azure Data Factory (ADF) and Azure Synapse Analytics.
• Hands-on experience with Delta Lake architecture and modern data lakehouse implementations.
• Advanced knowledge of Data Modeling and Data Warehousing concepts.
• Strong exposure to the Microsoft Fabric ecosystem.
• Proven expertise in designing cloud-native data architectures and enterprise-scale solutions.
• Strong design thinking and problem-solving capabilities.
• Working knowledge of CI/CD pipelines and DevOps practices.
• Experience in cloud cost optimization, performance tuning, and resource governance.

Key Responsibilities

• Lead and own the end-to-end delivery of enterprise data platforms, including architecture, development, deployment, and optimization.
• Define and drive the data platform roadmap, ensuring alignment with business objectives and strategic priorities.
• Architect and govern scalable data pipelines, data models, data warehouses, and lakehouse solutions using Azure services.
• Provide technical leadership to Data Engineering teams through design reviews, code reviews, and implementation of engineering best practices.
• Drive the adoption and implementation of Microsoft Fabric, APIs, Azure Functions, and Python-based solutions.
• Ensure platform scalability, reliability, security, and operational excellence while optimizing cloud costs.
• Establish and enforce data governance, data quality, security, and compliance standards.
• Implement and drive CI/CD, DevOps, and Agile delivery practices across the data engineering landscape.
• Collaborate with business stakeholders, architects, and technology teams to translate business requirements into scalable technical solutions.
• Mentor, coach, and develop Data Engineering talent, fostering a culture of continuous learning and innovation.
• Leverage GitHub Copilot and AI-assisted engineering tools to improve productivity and solution quality.
• Drive innovation by evaluating emerging technologies and identifying opportunities for platform modernization.

Experience

• 10+ years of overall IT experience, including 10+ years in Data Engineering.
• Minimum 5+ years of leadership experience managing large-scale Data Engineering programs and teams.
• Proven track record of leading cross-functional teams and delivering complex enterprise data transformation initiatives.Experience building, hiring, and scaling high-performing Data Engineering teams.
• Experience working within multi-vendor and globally distributed delivery environments.
• Strong experience in stakeholder management, delivery governance, and executive-level communication.

Preferred Skills

• Experience building and managing enterprise APIs and Azure Function Apps.
• Exposure to framework-based data engineering development approaches.
• Experience in solution architecture, stakeholder engagement, and delivery governance.
• Azure certifications such as:
• Azure Solutions Architect Expert
• Familiarity with MLOps, Data Science, and AI-driven analytics solutions.
• Hands-on experience with Power BI and enterprise reporting platforms.
• Experience leveraging AI and Generative AI capabilities within Data Engineering ecosystems.
• Strong ownership mindset with a focus on accountability and business outcomes.
• Excellent communication, stakeholder management, and influencing skills.
• Ability to balance technical leadership with delivery and people management responsibilities.
• Strategic thinker with a strong focus on execution excellence.
• Strong conflict resolution and cross-functional collaboration skills.
• Adaptability and resilience in fast-paced, evolving environments.
• Passion for mentoring teams and fostering a culture of innovation and continuous improvement.