AI Engineer, AI & Applications

Academic Key; Sydney; Híbrido; AUD 140.000 - 190.000; Jornada completa

Descripción de la vacante

Firmus Technologies seeks an experienced AI Engineer to design, develop, implement and maintain end-to-end MLOps and AIOps foundations for Firmus AI Factory, our AI platform, in Sydney. This role emphasizes trustworthy, repeatable and scalable model deployment with clear governance and automated gates.

You will own the model registry and promotion lifecycle using MLflow, establish reproducibility and lineage, build automated quality gates, define SLOs/SLIs, and create production dashboards.

Formación

• Bachelor’s degree or higher in Software Engineering, CS, IT or related field.
• 3+ years in MLOps or production ML delivery pipelines.
• Hands-on MLflow Model Registry with stages/aliases, governance and integration into release pipelines.
• Experience with CI/CD, deployment automation, monitoring and drift detection.
• Ability to design end-to-end MLOps workflows and maintain production ML services.

Responsabilidades

• Design, develop, implement and maintain end-to-end MLOps workflows from training to retraining in dev/staging/production.
• Own the model registry and promotion lifecycle using MLflow with environment separation and access controls.
• Establish reproducibility and lineage across the model lifecycle with versioned artifacts and datasets.
• Create automated model quality gates for production including accuracy, latency, cost and safety.
• Define SLOs/SLIs and develop production dashboards for monitoring model performance and health.
• Collaborate with ML engineers, data scientists and operations on CI/CD gates and incident response.

Conocimientos

MLOps

ML Platform

Software Engineering

Production Engineering

MLflow

CI/CD

Model Registry

Evaluation Systems

Monitoring

Incident Response

Distributed Systems

Educación

Bachelor’s degree or higher in Software Engineering / Computer Science / IT

Herramientas

MLflow

PagerDuty

Slack

Descripción del empleo

Role Summary Firmus Technologies is seeking an experienced AI Engineer to design, develop, implement and maintain the MLOps and AIOps foundations for Firmus AI Factory, our AI platform, to make it trustworthy, repeatable and scalable. This is a pioneering role where you will establish the end-to-end MLOps workflows, turning model development into a disciplined software release process with clear governance, automated evaluation gates and reliable promotion to production. You will also enable our Model Arena initiative by designing, developing and operationalising the evaluation pipelines and standards so model choices for RAG and agentic applications are data-driven, reproducible and production-safe. You are also responsible for the design, testing, maintenance and reliability of Firmus AI Factory AI features, including training jobs, inference services and RAG systems. You will define quality gates, model promotion workflows, production monitoring and incident response procedures. Your role is to ensure AI platform features are engineered to be as trustworthy as core infrastructure — fast, reliable, maintainable and observable. You will work across the team, partnering with engineers on CI/CD gates, data scientists on quality metrics, and operations teams on L2/L3 incident response. Key Responsibilities Design, develop, implement and maintain end-to-end MLOps workflows: training → evaluation → registry → deployment → monitoring → retraining/retirement in dev/staging/production environments, with clear standards and ownership boundaries. Own the model registry and promotion lifecycle using MLflow, including stage/alias strategy, approvals, environment separation, access control, rollback readiness and integration with software release pipelines. Establish reproducibility and lineage across the model lifecycle, including versioned code/configuration, artifact traceability, dataset/version references and repeatable evaluation runs. Design, develop and implement automated model quality gates for production, including accuracy, latency, cost, safety and reliability requirements. Define SLOs/SLIs for all AI platform features, including training job success rate, inference latency p99, RAG retrieval accuracy, availability and cost metrics. Develop and maintain production monitoring dashboards to track model performance, data drift and operational health, and integrate monitoring outputs with alerting tools such as PagerDuty and Slack. Create on-call runbooks and triage procedures for AI service incidents and lead postmortem-driven improvements to improve software reliability and maintainability. Instrument AI services for debugging, including request traces, GPU metrics per model, retrieval performance and communication bottlenecks. Integrate evaluation frameworks, including benchmarking, RAGAS and LLM-as-judge, into CI/CD pipelines to support automated testing, deployment and release management. Skills & Experience Bachelor’s degree or higher qualification in Software Engineering, Computer Science, Information Technology or a closely related field. 3 + years in MLOps, ML platform, software engineering or production engineering roles with hands-on ownership of production ML delivery pipelines. Deep understanding of the ML software lifecycle, including model versioning, promotion strategies, evaluation automation, governance, deployment strategies, monitoring and drift detection. Hands-on experience with MLflow Model Registry workflows, including stages/aliases, approvals, traceability and integration of registry actions into software release pipelines. Experience designing, developing and operationalising model evaluation systems, including metrics standards, orchestration, logging and reproducibility. Strong software engineering and production fundamentals, including metrics, logs, traces, alert design, incident response and reliability practices. Familiarity with CI/CD pipelines, model packaging, deployment automation and software testing, with the ability to collaborate across ML engineers, platform/SRE and application teams to turn requirements into robust workflows. Understanding of distributed systems, resource management and failure modes in training and inferencing environments. Key Competencies Production Ownership: comfortable owning services in production; proactive about monitoring, alerting, and preventing issues. Reliability Engineering: can define SLOs, error budgets, and blameless postmortem culture. Cross-Functional Leadership: works with ML engineers, data scientists, and platform teams; unblocks reliably. Incident Response: triage skills, root cause analysis, systemic thinking (not just fighting fires). Programmatic automation reduces toil and makes the right path with a balanced rigor with speed. Communication: explains complex ML/systems issues clearly to both technical and non-technical stakeholders. Location & Reporting Location Sydney, Australia Reporting to Head of AI &