AI Engineer

Kaizen Global Technologies; Sydney; On-site; AUD 120,000 - 160,000; Full time

Job summary

Kaizen Global Technologies is seeking an experienced Platform Engineer to own the operational lifecycle of AI Agents on GCP. You will oversee upgrades, evaluations, and maintenance of the Agent Runtime, while ensuring robust IAM and authentication integration with Okta.

You’ll implement monitoring, observability, and security controls across cloud services and collaborate with AI, Cloud, Security and Engineering teams.

Qualifications

• Experience with Google Cloud Platform (GCP).
• Hands-on with Cloud Run, Cloud SQL and Load Balancing.
• Experience with LLMs, AI Agents, Agent Runtime and LLM evaluation.
• Strong knowledge of IAM, Okta, authentication and cloud security.
• Experience with Cloud Monitoring, observability and metric analysis.
• Exposure to Braintrust or similar AI monitoring/evaluation platforms.
• Strong troubleshooting and stakeholder management skills.

Responsibilities

• Manage the operational lifecycle of AI Agents, including evaluations, upgrades and platform maintenance.
• Manage GCP infrastructure including Cloud Run, Cloud SQL, Agent Runtime and Load Balancers.
• Implement monitoring and observability solutions to track platform health, performance and AI metrics.
• Configure and maintain IAM, authentication and Okta integration.
• Troubleshoot platform and infrastructure issues and perform performance optimisation.
• Maintain secure cloud infrastructure following security and governance standards.
• Collaborate with AI, Cloud, Security and Engineering teams to manage shared platform services.

Skills

GCP expertise

Cloud Run

Cloud SQL

Load Balancing

LLMs & AI Agents

Agent Runtime

LLM evaluation

IAM & Okta

Cloud security

Monitoring & observability

Metrics analysis

Braintrust platform

Tools

Braintrust or similar AI monitoring platforms

Job description

• Manage the operational lifecycle of AI Agents, including LLM evaluations, upgrades and platform maintenance.
• Manage GCP infrastructure including Cloud Run, Cloud SQL, Agent Runtime and Load Balancers.
• Implement monitoring and observability solutions to track platform health, performance and AI-specific metrics.
• Configure and maintain IAM, authentication and Okta integration.
• Troubleshoot platform and infrastructure issues and perform performance optimisation.
• Maintain secure cloud infrastructure following security and governance standards.
• Collaborate with AI, Cloud, Security and Engineering teams to manage shared platform services.

Key Skills

• Strong hands-on experience with Google Cloud Platform (GCP).
• Experience with Cloud Run, Cloud SQL and Load Balancing.
• Experience with LLMs, AI Agents, Agent Runtime and LLM evaluation.
• Strong knowledge of IAM, Okta, authentication and cloud security.
• Experience with Cloud Monitoring, observability and metric analysis.
• Exposure to Braintrust or similar AI monitoring/evaluation platforms.
• Strong troubleshooting, communication and stakeholder management skills.