Data Engineer

Compare the Market Australia; Brisbane; Hybrid; AUD 120,000 - 170,000; Full time

Benefits offered by this job

Flexible work arrangements

ME leave and Volunteer Day leave

Annual celebrations and family events

Free coffee on Mondays

Job summary

Compare the Market Australia (iSelect and Compare the Market) is hiring a Data Engineer to design, build and maintain scalable cloud data solutions. You will deliver robust data pipelines, lakehouse architectures and automation frameworks to support analytics and AI/ML initiatives.

Join a collaborative team that embraces flexible work arrangements and values innovation, data governance, and cross-functional partnership to drive impactful insights.

Qualifications

• Degree in Computer Science, Analytics, IT or related field, or equivalent practical experience.
• 3+ years in Data Engineering or 5+ years across engineering, data or BI.
• 3+ years with cloud platforms and lakehouse technologies (Databricks, Snowflake, BigQuery, Redshift or Synapse).
• Experience building batch and near real-time pipelines using Spark, Python or SQL.
• Proficiency with ETL/ELT, DataOps and orchestration tools (DBT, Airflow, Dagster, Kestra).
• Knowledge of data modelling, governance, lineage, and access management.
• CI/CD, IaC, APIs and automation frameworks (Azure DevOps, GitHub).
• Exposure to GenAI and AI-related data workflows is a bonus.

Responsibilities

• Design and deliver scalable data solutions aligned with enterprise architecture.
• Build and optimise end-to-end data pipelines and integrations.
• Implement, test and deploy solutions using IaC and modern release processes.
• Ensure data quality and reliability through monitoring and logging.
• Collaborate with architects, BI developers and stakeholders to optimise platforms.
• Work in an Agile environment with strong communication and data lifecycle understanding.
• Apply DataOps and automation to improve performance and scalability.

Skills

3+ years in Data Engineering

5+ years across engineering, data orBI

Batch and near real-time pipelines

ETL/ELT and DataOps

CI/CD, IaC and automation

Data modelling, governance, lineage

Spark, Python, SQL

Cloud platforms (AWS/Azure/GCP)

Databricks, lakehouse architectures

GenAI exposure (optional)

Education

Degree in Computer Science, Analytics, IT or related field

Tools

Databricks

Snowflake

BigQuery

Redshift

Synapse

Airflow

DBT

DLT

Dagster

Kestra

Azure DevOps

GitHub

Job description

Who We Are

iSelect and Compare the Market are trusted, well-known brands dedicated to helping millions of Australians compare and buy personal finance and household products such as insurance, energy, and loans. Together, we provide a seamless, customer-focused experience that makes it easier for Australians to find the right products to suit their needs.

What You’ll Be Doing

As a Data Engineer, you’ll design, build and maintain scalable, secure cloud data solutions using Databricks and related platforms. You’ll deliver robust data pipelines, lakehouse architectures, automation frameworks and curated data assets that support reporting, analytics and AI/ML initiatives.

• Design and deliver scalable data solutions aligned with enterprise architecture and governance standards.
• Build and optimise end-to-end data pipelines, integrations and workflows.
• Implement, test and deploy solutions using IaC, engineering best practices and release processes.
• Ensure data quality and reliability through monitoring, logging, issue remediation and proactive risk management.
• Partner with Solution Architects, Data Engineers, BI Developers and business stakeholders to optimise platforms and support analytics.
• Work effectively in an Agile environment, bringing strong communication, business knowledge and an end-to-end understanding of the Data & Insights lifecycle.
• Apply data engineering best practices and advance DataOps and automation to continually improve platform performance and scalability.

What’s In It For You

At Compare the Market and iSelect, we’re all about helping people make smart choices – and joining us might just be yours. We offer an environment where your ideas are heard, your contributions are valued, and your growth genuinely matters. Collaboration is part of our DNA and you’ll be joining a supportive, forward-thinking team where you can do meaningful work (and be recognised for it!)

We’re guided by three shared values that shape how we work, how we treat each other, and how we show up for our customers - Drive, Noble Intent and All Together.

• Vibrant and social community who have annual celebrations, family fun days and regular events
• Enjoy flexible work arrangements, including the option of one day working from home each week or a 9-day fortnight
• Enjoy additional leave days - ‘ME’ leave and ‘Volunteer Day’ leave
• Free coffee on Mondays
• Option to purchase additional leave
• Return To Work bonus for returning parents
• Quarterly Award and Recognition program

Qualifications

What Sets You Apart

• Degree in Computer Science, Analytics, IT or a related field, or equivalent practical experience.

Experience & Skills

• 3+ years in Data Engineering, or 5+ years across engineering, data or BI.
• 3+ years with cloud platforms such as AWS, Azure or GCP and Lakehouse technologies including Databricks, Snowflake, BigQuery, Redshift or Synapse.
• Building batch and near real-time pipelines using Spark, Python, SQL or similar languages.
• ETL/ELT, DataOps and orchestration tools such as DBT, DLT, Airflow, Dagster or Kestra.
• Cloud storage and Lakehouse architectures, including S3, ADLS or HDFS.
• Data modelling and governance, including Data Vault, metadata, lineage, auditability and access management.
• CI/CD, IaC, APIs, Azure DevOps, GitHub and automation frameworks.
• Enterprise Data Warehouse or Hadoop experience is advantageous.
• Exposure to GenAI, including RAG, AI Agents, Semantic/Vector Search, Knowledge Graphs, Embeddings, Fine-Tuning, Model Adaptation or AI Guardrails, is highly regarded.

At Compare the Market and iSelect we’re proud to be an Equal Opportunity Employer and we embrace diversity.

This role is available to candidates with unlimited Australian working rights only. Sponsorship is not available.

iSelect and Compare the Market, brands of iCompare Holdings, are committed to protecting your privacy. All applications and personal information are kept confidential and securely stored in line with Australian privacy laws, used only for recruitment purposes, and not shared with third parties outside iCompare Holdings without your consent.