AI Data Engineer

Compare the Market Australia; Brisbane; On-site; AUD 140,000 - 190,000; Full time

Benefits offered by this job

Flexible work arrangements

Annual celebrations & events

Volunteer Day leave

Free coffee on Mondays

Leave purchase options

Return To Work bonus

Quarterly awards program

Job summary

Compare the Market Australia (iSelect and CTM brands) is seeking an AI Data Engineer to design and deliver AI‑enabled data platform capabilities. You will build scalable, secure foundations to accelerate agentic AI adoption across the enterprise.

You will work across cloud‑based data platforms to develop AI‑driven infrastructure, automation, and services supporting ML, Generative AI, and advanced analytics for better data discovery and decision making.

Qualifications

• 4+ years' experience in Gen AI Engineering, Data Engineering, Integration Engineering, Data Science, Software Engineering, or related disciplines, delivering enterprise‑scale solutions from concept through to commercial deployment
• Proven experience in designing, developing, deploying, monitoring, and optimising raw models, including model training, fine‑tuning, evaluation, versioning, experiment tracking, observability, drift detection, and token usage management
• Strong expertise in Generative or Agentic AI engineering patterns, including Retrieval‑Augmented Generation (RAG), semantic search, vector search, embeddings, prompt engineering, context management, AI agents, tool calling, and multi‑agent architectures
• Experience building agent powered applications and intelligent automation solutions, including enterprise copilots, chatbots, knowledge assistants, research agents, workflow automation, and decision‑support capabilities
• Hands‑on experience with modern frameworks and platforms, including PyTorch, TensorFlow, Scikit‑Learn, Hugging Face, LangChain, LangGraph, LlamaIndex, Semantic Kernel, CrewAI, AutoGen, and Microsoft Copilot Studio
• Experience with AI‑driven data solutions on EDPs, knowledge repositories, feature stores, vector databases, and data products using cloud‑native Data platforms such as Databricks, Snowflake, Fabric, Synapse, Redshift, BigQuery, AWS, Azure, or GCP
• Experience operationalising agent solutions using MLOps and LLMOps practices, including MLflow, Databricks ML, SageMaker, Bedrock, AI Foundry, Vertex AI, containerisation, and deployment automation
• Strong understanding of AI governance, security, and Responsible AI principles, including data privacy, data sovereignty, model governance, access controls, metadata management, lineage, compliance, and optimisation of model performance, scalability, latency, and cost efficiency

Responsibilities

• Design, develop, and implement AI‑driven data solutions and agent‑based products on the EDW that support model learning, generative AI, intelligent automation, and advanced analytics for better data discovery
• Develop, deploy, and operationalise foundational models, LLMs, RAG, semantic search, vector retrieval, embeddings, and enterprise knowledge management capabilities
• Design and implement intelligent automation and agentic AI solutions leveraging AI agents, orchestration frameworks, multi‑agent architectures, workflow automation, and reusable AI services and APIs
• Establish and champion AI engineering best practices including MLOps, LLMOps, model monitoring, query observability, experiment tracking, benchmarking and token usage management
• Drive AI innovation, platform optimisation, and governance by evaluating emerging technologies, improving model performance and cost efficiency, and ensuring compliance with security, privacy, data governance, and Responsible AI standards

Skills

Gen AI Engineering

Data Engineering

Integration Engineering

Data Science

Software Engineering

MLOps

AI Governance

Education

Bachelor's degree in Computer Science / Mathematics / Data Science

Tools

PyTorch

TensorFlow

Scikit-Learn

Hugging Face

LangChain

LangGraph

LlamaIndex

Semantic Kernel

CrewAI

AutoGen

Microsoft Copilot Studio

Job description

Who We Are

iSelect and Compare the Market are trusted, well‑known brands dedicated to helping millions of Australians compare and buy personal finance and household products such as insurance, energy, and loans. Together, we provide a seamless, customer‑focused experience that makes it easier for Australians to find the right products to suit their needs.

What You’ll Be Doing

The AI Data Engineer designs and delivers AI‑enabled data platform capabilities that position CTM at the forefront of data, automation, and AI. Combining data, AI, and platform engineering disciplines, the role builds scalable, secure, and production‑ready foundations that accelerate the adoption of agentic AI across the organisation.

Working across CTM’s cloud‑based data platform, the role develops AI‑driven infrastructure, intelligent automation, and reusable AI services that support advanced analytics, machine learning, Generative AI, and agent‑based technologies. The AI Data Engineer ensures CTM’s data ecosystem is AI‑ready through modern architectures, automation frameworks, vector‑based retrieval, and intelligent agent capabilities.

Partnering with data teams, architects, and business stakeholders, the role identifies opportunities to improve efficiency, automate processes, enhance decision‑making, and accelerate the delivery of data products and insights. The role also contributes to engineering standards, AI governance, and platform capabilities that enable the secure, responsible, and sustainable adoption of AI across the enterprise.

• Design, develop, and implement AI‑driven data solutions and agent‑based products on the EDW that support model learning, generative AI, intelligent automation, and advanced analytics for better data discovery
• Develop, deploy, and operationalise foundational models, LLMs, RAG, semantic search, vector retrieval, embeddings, and enterprise knowledge management capabilities
• Design and implement intelligent automation and agentic AI solutions leveraging AI agents, orchestration frameworks, multi‑agent architectures, workflow automation, and reusable AI services and APIs
• Establish and champion AI engineering best practices including MLOps, LLMOps, model monitoring, query observability, experiment tracking, benchmarking and token usage management
• Drive AI innovation, platform optimisation, and governance by evaluating emerging technologies, improving model performance and cost efficiency, and ensuring compliance with security, privacy, data governance, and Responsible AI standards

What’s In It For You

At Compare the Market and iSelect, we’re all about helping people make smart choices – and joining us might just be yours. We offer an environment where your ideas are heard, your contributions are valued, and your growth genuinely matters. Collaboration is part of our DNA and you’ll be joining a supportive, forward‑thinking team where you can do meaningful work (and be recognised for it!)

We’re guided by three shared values that shape how we work, how we treat each other, and how we show up for our customers – Drive, Noble Intent and All Together.

Plus, we offer a range of great perks to help you thrive both in and out of work:

• Vibrant and social community who have annual celebrations, family fun days and regular events
• Enjoy flexible work arrangements, including the option of one day working from home each week or a 9‑day fortnight
• Enjoy additional leave days – ‘ME’ leave and ‘Volunteer Day’ leave
• Free coffee on Mondays
• Option to purchase additional leave
• Return To Work bonus for returning parents
• Quarterly Award and Recognition program

What Sets You Apart

Minimum a bachelor’s degree qualified in Computer Science, Mathematics, Data Science or a relevant discipline

Experience & Skills

• 4+ years' experience in Gen AI Engineering, Data Engineering, Integration Engineering, Data Science, Software Engineering, or related disciplines, delivering enterprise‑scale solutions from concept through to commerical deployment
• Proven experience in designing, developing, deploying, monitoring, and optimising raw models, including model training, fine‑tuning, evaluation, versioning, experiment tracking, observability, drift detection, and token usage management
• Strong expertise in Generative or Agentic AI engineering patterns, including Retrieval‑Augmented Generation (RAG), semantic search, vector search, embeddings, prompt engineering, context management, AI agents, tool calling, and multi‑agent architectures
• Experience building agent powered applications and intelligent automation solutions, including enterprise copilots, chatbots, knowledge assistants, research agents, workflow automation, and decision‑support capabilities
• Hands‑on experience with modern frameworks and platforms, including PyTorch, TensorFlow, Scikit‑Learn, Hugging Face, LangChain, LangGraph, LlamaIndex, Semantic Kernel, CrewAI, AutoGen, and Microsoft Copilot Studio
• Experience with AI‑driven data solutions on EDPs, knowledge repositories, feature stores, vector databases, and data products using cloud‑native Data platforms such as Databricks, Snowflake, Fabric, Synapse, Redshift, BigQuery, AWS, Azure, or GCP
• Experience operationalising agent solutions using MLOps and LLMOps practices, including MLflow, Databricks ML, SageMaker, Bedrock, AI Foundry, Vertex AI, containerisation, and deployment automation
• Strong understanding of AI governance, security, and Responsible AI principles, including data privacy, data sovereignty, model governance, access controls, metadata management, lineage, compliance, and optimisation of model performance, scalability, latency, and cost efficiency

At Compare the Market and iSelect we’re proud to be an Equal Opportunity Employer and we embrace diversity.

This role is available to candidates with unlimited Australian working rights only. Sponsorship is not available.

iSelect and Compare the Market, brands of iCompare Holdings, are committed to protecting your privacy. All applications and personal information are kept confidential and securely stored in line with Australian privacy laws, used only for recruitment purposes, and not shared with third parties outside iCompare Holdings without your consent.