Lead Data Engineer: AI & Data Platform, Baseline Clearance

Clicks IT Recruitment; City of Melbourne; On-site; AUD 180,000 - 230,000; Full time

Job summary

Clicks IT Recruitment is seeking a Lead Data Engineer in Australia to design, build and maintain scalable data and AI infrastructure supporting ML, NLP and Generative AI workloads. You will work within a multidisciplinary team of data scientists and policy experts across nationwide operations.

The role requires experience with scalable data pipelines, cloud-based environments and advanced Python development, plus a Baseline security clearance to apply.

Qualifications

• Experience designing, building and maintaining scalable data pipelines and data platforms.
• Strong expertise in data ingestion, transformation, automation and bulk processing workflows.
• Experience working with cloud-based and distributed data environments.
• Advanced Python development skills.
• Proven experience deploying ML, NLP and/or Generative AI solutions into production.
• Experience supporting end-to-end delivery of AI-enabled products and services.

Responsibilities

• Designing, developing and maintaining scalable data ingestion, transformation and processing pipelines for structured and unstructured data sources.
• Applying modern data engineering practices to support collection, preparation, cleansing, validation and integration of skills-related datasets.
• Developing automated data quality, testing and validation capabilities that support reliable and reproducible analytical outcomes.
• Designing and implementing repeatable validation pipelines that support the assessment, monitoring and quality assurance of AI and NLP model outputs.
• Collaborating with Data Scientists to operationalise machine learning, NLP and Generative AI solutions within scalable production environments.
• Developing and maintaining model-serving, deployment and API integration capabilities that support analytical products and downstream systems.
• Implementing data governance, metadata management, lineage tracking and versioning practices to support transparency, traceability and reproducibility.
• Developing and maintaining cloud-based and distributed data processing environments capable of supporting large-scale data and AI workloads.

Skills

Data pipelines

Python

Cloud platforms

ML/AI deployment

Data governance

Automation / ETL

NLP / Generative AI

Job description

Clicks IT Recruitment is seeking a Lead Data Engineer in Australia to design, build and maintain scalable data and AI infrastructure supporting ML, NLP and Generative AI workloads. You will work within a multidisciplinary team of data scientists and policy experts across nationwide operations.

The role requires experience with scalable data pipelines, cloud-based environments and advanced Python development, plus a Baseline security clearance to apply.