Senior GCP DevOps & LLMOps Engineer

Tredence Inc.; Bengaluru; On-site; INR 4,200,000 - 6,600,000; Full time

Job summary

Tredence Inc. is seeking a Senior GCP DevOps & LLMOps Engineer to architect, deploy, and maintain robust infrastructure for enterprise-scale LLM and Generative AI apps in Bengaluru.

You will bridge traditional DevOps with ML ops, enabling automated deployment, scaling, and operation of AI workloads. You will lead IaC and security across GCP, craft resilient CI/CD pipelines, and implement end-to-end LLM lifecycles, including vector databases and model serving engines.

Qualifications

• 5+ years in DevOps, SRE, or Cloud Platform Engineering with at least 3+ years on Google Cloud Platform (GCP).
• Terraform advanced proficiency (state management, reusable modules, workspaces).
• 1.5+ years of practical LLM/ML ops experience with Vertex AI or Kubernetes-based model serving.
• Strong containerization with Docker and GKE, including GPU node pools and autoscaling.
• Hands-on CI/CD with GitOps patterns (ArgoCD, Flux) or GitHub Actions/Cloud Build.
• Python and Bash/Shell scripting proficiency; security and networking basics in cloud infra.

Responsibilities

• Design, build, and maintain production GCP environments using modular Terraform/ Terragrunt code.
• Manage multi-environment cloud resources (VPC, Private Service Connect, Cloud Interconnect, IAM, KMS, Cloud Storage).
• Implement automated LLM/ML lifecycle pipelines (prompt management, fine-tuning, RAG, evaluation harness).
• Deploy and scale vector stores (Vertex AI Vector Search, Pinecone, Qdrant, pgvector) with monitoring.
• Establish automated testing for infrastructure and model release pipelines (canary, blue/green).
• Set up observability stacks (Google Cloud Monitoring, OpenTelemetry, Prometheus, Grafana) and track SLIs/SLOs.

Skills

Terraform

GCP

CI/CD

Python

Bash/Shell

OpenTelemetry

Prometheus

Grafana

Tools

Terragrunt

OpenTofu

TFLint

GKE

Docker

Helm

ArgoCD

GitHub Actions

Cloud Build

GitLab CI

Vertex AI

LangSmith

Phoenix

Qdrant

pgvector

Job description

Job Description: Senior GCP DevOps & LLMOps Engineer

Role Summary

We are looking for an experienced Senior GCP DevOps & LLMOps Engineer to architect, deploy, and maintain robust infrastructure supporting enterprise-scale Large Language Model (LLM) and Generative AI applications. In this role, you will bridge the gap between traditional DevOps, platform engineering, and modern machine learning operations.

You will lead the automation of Google Cloud Platform (GCP) environments using Terraform, design resilient CI/CD pipelines, and implement production-grade LLMOps architectures--including model serving, inference routing, vector database orchestration, evaluation pipelines, and GPU/TPU resource optimization.

Key Responsibilities

1. Infrastructure as Code (IaC) & Cloud Architecture

• Design, build, and maintainproduction GCP environments using modular, reusable Terraform (and Terragrunt) code.

• Manage multi-environment cloud resources across VPC networking, Private Service Connect, Cloud Interconnect, IAM, KMS, and Cloud Storage.

• Enforce security best practices, least-privilege IAM policies, Workload Identity Federation, and automated compliance auditing.

2. LLMOps & GenAI Platform Engineering

• Build automated deployment, scaling, and operational infrastructure for generative AI models hosted on Vertex AI, GKE (Google Kubernetes Engine), or hybrid inference servers.

• Implement or Understanding end-to-end LLM lifecycle pipelines: prompt management, fine-tuning workloads, automated retrieval-augmented generation (RAG) components, and evaluation harness pipelines.

• Deploy, manage, and scale vector databases (e.g., Vertex AI Vector Search, Pinecone, Qdrant, pgvector) with proper indexing, backup, and latency monitoring.

• Architect and optimize model-serving engines.

3. CI/CD & Pipeline Automation

• Build and maintain scalable CI/CD pipelines using Cloud Build, GitHub Actions, or GitLab CI for both microservices and machine learning models.

• Establish automated testing for infrastructure changes (e.g., tflint, tfsec, terratest) and model release pipelines (canary deployments, blue/green rollouts, shadow traffic).

• Package and deploy containerized AI/ML applications using Docker, Artifact Registry, and Helm charts on GKE.

4. Observability, Guardrails & Reliability

• Set up deep observability stacks using Google Cloud Monitoring, OpenTelemetry, Prometheus, and Grafana to track standard SRE metrics (SLIs/SLOs, latency, throughput).

• Implement LLM-specific telemetry: token consumption rates, inference latency, model drift, cost-per-query, and hallucinations/groundedness metrics using tools such as LangSmith, Phoenix/Arize, or Vertex AI Model Monitoring.

• Integrate safety guardrails, prompt sanitization, rate limiting, and fallback/resiliency mechanisms across inference endpoints.

5. FinOps & Compute Optimization

• Monitor and optimize cloud spending for compute-intensive workloads, specifically LLM quota monitoring, spot instances, and dynamic autoscaling.

• Partner with engineering and product teams to project infrastructure costs and establish budget s and quota guardrails.

Requirements & Qualifications

Must-Have

• Experience: 5+ years in DevOps, SRE, or Cloud Platform Engineering with at least 3+ years dedicated to Google Cloud Platform (GCP).

• Terraform Expertise: Advanced proficiency with Terraform (state management, reusable module design, workspaces, policy-as-code with Sentinel or OPA).

• LLMOps / MLOps Background: 1.5+ years of practical experience operationalizing LLMs or ML workloads, including familiarization with Vertex AI (Model Garden, Pipelines, Endpoints) or custom model serving on Kubernetes.

• Containerization & Orchestration: Strong background in Docker and Google Kubernetes Engine (GKE), including cluster setup, GPU node pools, ingress controllers, and autoscaling (HPA, KEDA).

• CI/CD: Hands-on experience configuring automated pipelines with GitOps patterns (e.g., ArgoCD, Flux) or modern CI engines (GitHub Actions, Cloud Build).

• Scripting & Languages: Solid programming/scripting proficiency in Python and Bash/Shell.

• Security & Networking: Strong understanding of GCP VPC architecture, private endpoints, service perimeters (VPC-SC), and Secret Manager.

Nice-to-Have

• Experience deploying agentic frameworks (e.g., LangChain, LlamaIndex, LangGraph, CrewAI) or Model Context Protocol (MCP) servers in production.

• GCP Certifications (e.g., Google Cloud Certified Professional Cloud DevOps Engineer, Professional Cloud Architect, or Professional Machine Learning Engineer).

• Familiarity with distributed model training and high-throughput inference caching strategies (Redis, Semantic Cache).

• Exposure to regulatory compliance frameworks (SOC2, HIPAA, ISO 27001) in relation to cloud AI systems.

Technical Stack Overview

Domain

Technologies & Tools

Cloud Provider

Google Cloud Platform (GCP)

Infrastructure as Code

Terraform, Terragrunt, OpenTofu, TFLint

Containers & Orchestration

GKE, Docker, Helm, ArgoCD

AI / LLMOps

Vertex AI, vLLM, Triton, Ray, Hugging Face, LangSmith / Phoenix

Vector Search & DBs

Vertex AI Vector Search, Qdrant, Pinecone, Cloud SQL (pgvector)

CI/CD & Code Quality

GitHub Actions, Cloud Build, GitLab CI, Pre-commit, SonarQube

Observability & Ops

Google Cloud Monitoring, Prometheus, Grafana, OpenTelemetry

Scripting & Automation

Python, Bash

Required Skills