AI Platform Engineer

Umbrella Club; Sydney; On-site; AUD 120,000 - 180,000; Full time

Job summary

Umbrella Club seeks an experienced AI Platform / Agent Operations Engineer to support the operation, security and performance of our enterprise AI agent platform running on GCP. You will manage agent lifecycle, upgrades and reliability across Cloud Run, Cloud SQL and runtime environments.

Responsibilities include implementing monitoring and observability, assisting with LLM evaluation and tuning, and maintaining IAM controls including Okta.

Qualifications

• Experience in AI/LLM platform operations or GenAI environments.
• Strong hands-on experience with GCP, particularly Cloud Run, Cloud SQL and Load Balancing.
• Knowledge of LLM evaluation, AI agent platforms and Agent Runtime.
• Experience with monitoring and observability tools.
• Experience with Okta / Identity & Access Management.
• Strong understanding of cloud security and operational best practices.

Responsibilities

• Manage the operational lifecycle of AI agents and LLM applications, including evaluations and version upgrades.
• Manage GCP infrastructure including Cloud Run, Cloud SQL, Load Balancing and Agent Runtime.
• Implement monitoring and observability to track platform and AI performance.
• Support LLM evaluation, performance tuning and issue resolution.
• Manage identity and access controls, including Okta integration.
• Work with cross-functional teams on shared services, security and platform governance.

Skills

GCP core services

LLM ops

Monitoring & observability

IAM / Okta

Cloud security

Agent Runtime

Tools

Cloud Run

Cloud SQL

Load Balancing

Okta

Job description

We are seeking an experienced AI Platform / Agent Operations Engineer to support the operation, security and performance of an enterprise AI agent platform running on GCP.

Key Responsibilities:

• Manage the operational lifecycle of AI agents and LLM applications, including evaluations and version upgrades.
• Manage GCP infrastructure including Cloud Run, Cloud SQL, Load Balancing and Agent Runtime.
• Implement monitoring and observability to track platform and AI performance.
• Support LLM evaluation, performance tuning and issue resolution.
• Manage identity and access controls, including Okta integration.
• Work with cross-functional teams on shared services, security and platform governance.

Key Requirements:

• Strong hands-on experience with GCP, particularly Cloud Run, Cloud SQL and Load Balancing.
• Experience in AI/LLM platform operations or GenAI environments.
• Knowledge of LLM evaluation, AI agent platforms and Agent Runtime.
• Experience with monitoring and observability tools.
• Experience with Okta / Identity & Access Management.
• Strong understanding of cloud security and operational best practices.
• Desirable: Experience with Braintrust, Agentic AI, LLM observability and enterprise AI platforms.