Lead Data Engineer

Compare The Market; Brisbane; Hybrid; AUD 180,000 - 260,000; Full time

Benefits offered by this job

WFH one day per week

ME leave

Volunteer Day leave

Free coffee Mondays

Extra leave purchase

Return To Work bonus

Quarterly awards

Job summary

Compare the Market seeks a Lead Data Engineer to provide technical leadership for our Data & Analytics platforms. You will shape the data technology stack and establish engineering standards to deliver scalable, secure data solutions that power reporting, analytics, AI and business innovation.

You will champion DataOps, MLOps, AI engineering capabilities, and guide governance across our cloud-based data warehouse, lakehouse, and related platforms, ensuring reliability, security and high

Qualifications

• Bachelor’s degree in Computer Science, Analytics, IT or a relevant discipline.
• 10+ years in Data Engineering/Analytics or BI environments with enterprise-scale initiatives.
• Experience designing and governing cloud data platforms (data warehouses, lakehouses).

Responsibilities

• Provide technical leadership for CTM's Data & Analytics platforms and engineering practices.
• Oversee design, governance, and optimisation of cloud-based data warehouse, lakehouse, and AI-enabled platforms.
• Mentor engineers, drive DataOps, SecOps, CI/CD, IaC, and platform automation.

Skills

Data engineering

Cloud platforms

GenAI

DataOps

SecOps

CI/CD

IaC

Observability

Stakeholder mgmt

Mentoring

Education

Bachelor’s degree in Computer Science / Analytics / IT

Tools

Databricks

Snowflake

Synapse

BigQuery

Redshift

Spark

AWS/Azure/GCP

Job description

Who We Are

iSelect and Compare the Market are trusted, well-known brands dedicated to helping millions of Australians compare and buy personal finance and household products such as insurance, energy, and loans. Together, we provide a seamless, customer-focused experience that makes it easier for Australians to find the right products to suit their needs.

What You'll Be Doing

Reporting to the Data Engineering Manager, the Lead Data Engineer provides technical leadership for CTM's Data and Analytics platforms, engineering practices, and technology capabilities. The role is responsible for shaping the enterprise data technology stack, establishing engineering standards, and delivering scalable, secure, and high-performing data solutions that support reporting, analytics, AI, and business innovation.

As the organisation's technical SME for data engineering, the Lead Data Engineer drives the design, governance, and optimisation of cloud-based data warehouse, lakehouse, and AI-enabled platforms. Working closely with engineers, analysts, architects, and business stakeholders, the role champions modern engineering practices, platform automation, DataOps, MLOps, and AI engineering capabilities to ensure the data ecosystem remains robust, future-ready, and aligned to strategic objectives.

The role oversees the technical health, security, performance, scalability, and continuous improvement of the data platform landscape while fostering a culture of engineering excellence, innovation, and knowledge sharing. It also provides technical mentorship and governance across the Data and Analytics team, guiding solution design, architecture standards, and development practices to ensure the consistent delivery of high-quality data products and services.

• Provide technical leadership and architectural oversight across the Data & Analytics function, ensuring solutions align with enterprise architecture, security, scalability, and business objectives.
• Own and optimise the analytics ecosystem, including cloud data platforms, lakehouse environments, data integration, governance, and AI readiness.
• Lead the design and delivery of scalable data products, advanced analytics, and GenAI-enabled solutions that support self-service insights and future AI capabilities.
• Drive engineering excellence through standards, frameworks, DataOps, SecOps, CI/CD, Infrastructure as Code, automated testing, observability, and automation.
• Ensure platform reliability, security, governance and compliance through robust controls for data quality, lineage, metadata, access management, monitoring and privacy.
• Mentor engineers and business users while partnering with stakeholders to drive innovation, platform optimisation, emerging technology adoption, and data and AI maturity.

What's In It For You

At Compare the Market and iSelect, we’re all about helping people make smart choices – and joining us might just be yours.We offer an environment where your ideas are heard, your contributions are valued, and your growth genuinely matters. Collaboration is part of our DNA and you’ll be joining a supportive, forward-thinking team where you can do meaningful work (and be recognised for it!)

We’re guided by three shared values that shape how we work, how we treat each other, and how we show up for our customers - Drive, Noble Intent and All Together.

Plus, we offer a range of great perks to help you thrive both in and out of work:

• Vibrant and social community who have annual celebrations, family fun days and regular events
• Enjoy flexible work arrangements, including the option of one day working from home each week or a 9-day fortnight
• Enjoy additional leave days - ‘ME’ leave and ‘Volunteer Day’ leave
• Free coffee on Mondays
• Option to purchase additional leave
• Return To Work bonus for returning parents
• Quarterly Award and Recognition program

What Sets You Apart

Qualifications:

• Minimum a bachelor’s degree qualified in Computer Science, Analytics, IT or a relevant discipline

Experience & Skills:

• 10+ years' experience in Data Engineering, Analytics, Software Engineering, or Business Intelligence environments, including leading of enterprise-scale data initiatives
• Extensive experience designing, building, and governing cloud-based data platforms across AWS, Azure, or GCP, including data warehouses, lakehouses, and modern data ecosystems
• Strong expertise in data architecture and platform design, including Data Warehouses, Data Lakes, Lakehouses, Data Mesh, Digital win, Medallion Architecture, or scalable enterprise data products.
• Advanced experience with modern data platforms and technologies, including Databricks, Snowflake, Microsoft Fabric, Synapse, Redshift, BigQuery, Spark, streaming technologies, and cloud-native data services.
• Proven capability implementing engineering excellence practices, including DataOps, SecOps, CI/CD, IaC, DevOps, automated testing, observability, platform automation, and operational resilience.
• Deep understanding of enterprise data governance and management disciplines, including data quality, metadata, lineage, cataloguing, master data, security, privacy, access controls, audit, and regulatory compliance
• Experience leading data integration, automation, and platform modernisation initiatives, including APIs, event-driven architectures, reusable frameworks, orchestration platforms, and enterprise application integration
• Strong knowledge of designing infrastructure for AI eccentric solutions on EDWs, including Generative AI, RAG, vector search, embeddings, AI agents, intelligent automation, MLOps, and AI governance principles
• Demonstrated experience defining technology roadmaps, engineering standards, architecture patterns, up-and-coming AI models on the market and strategic platform investments that improve scalability, reliability, security, performance, and cost efficiency
• Proven leadership, stakeholder management, and mentoring capability, with experience developing engineering teams, providing technical direction, influencing strategic outcomes, and driving continuous improvement across Data & Analytics functions ad BAU initiatives.

At Compare the Market and iSelect we’re proud to be an Equal Opportunity Employer and we embrace diversity.

This role is available to candidates with unlimited Australian working rights only. Sponsorship is not available.

iSelect and Compare the Market, brands of iCompare Holdings, are committed to protecting your privacy. All applications and personal information are kept confidential and securely stored in line with Australian privacy laws, used only for recruitment purposes, and not shared with third parties outside iCompare Holdings without your consent.