Senior GCP DevOps Engineer

Tredence Inc.; Pune District, Chennai, Bengaluru; Hybrid; INR 3,000,000 - 6,000,000; Full time

Job summary

Tredence Inc. is seeking an experienced Senior GCP DevOps & LLMOps Engineer in Pune to architect and operate scalable cloud infrastructure for enterprise AI workloads.

You will bridge traditional DevOps with ML operations, automating deployment, security, and cost optimization across Vertex AI, GKE, and hybrid inference setups. You will lead IaC, CI/CD, and observability efforts, building robust pipelines, guarding against drift, and ensuring reliable model serving and data access.

Qualifications

• 5+ years in DevOps/SRE or Cloud Platform Engineering, with 3+ years on GCP
• Advanced Terraform with modules, state, and policy-as-code
• 1.5+ years of practical LLMs/ML workloads experience including Vertex AI or Kubernetes serving
• Strong containerization and GKE expertise with GPU node pools
• CI/CD automation with modern CI engines and GitOps patterns
• Python and Bash scripting proficiency, solid security basics on GCP

Responsibilities

• Design and maintain production GCP environments using Terraform and Terragrunt
• Build automated deployment, scaling, and ops infra for generative AI models
• Implement end-to-end LLM lifecycle pipelines (RAG, eval, model serving)
• Deploy and scale vector databases with proper indexing and monitoring
• Create scalable CI/CD pipelines for microservices and ML models
• Ensure observability and guardrails with SRE metrics and telemetry

Skills

GCP

Terraform

LLMOps

Kubernetes

Docker

CI/CD

Python

Bash

Tools

Terragrunt

GitHub Actions

Cloud Build

ArgoCD

Flux

Helm

Job description

Job Description: Senior GCP DevOps & LLMOps Engineer

Title: Senior DevOps / Platform Engineer (GCP & LLMOps)

Role Summary

We are looking for an experienced Senior GCP DevOps & LLMOps Engineer to architect, deploy, and maintain robust infrastructure supporting enterprise-scale Large Language Model (LLM) and Generative AI applications. In this role, you will bridge the gap between traditional DevOps, platform engineering, and modern machine learning operations.

You will lead the automation of Google Cloud Platform (GCP) environments using Terraform, design resilient CI/CD pipelines, and implement production-grade LLMOps architecturesincluding model serving, inference routing, vector database orchestration, evaluation pipelines, and GPU/TPU resource optimization.

Key Responsibilities

1. Infrastructure as Code (IaC) & Cloud Architecture

• Design, build, and maintain production GCP environments using modular, reusable Terraform (and Terragrunt) code.
• Manage multi-environment cloud resources across VPC networking, Private Service Connect, Cloud Interconnect, IAM, KMS, and Cloud Storage.
• Enforce security best practices, least-privilege IAM policies, Workload Identity Federation, and automated compliance auditing.

2. LLMOps & GenAI Platform Engineering

• Build automated deployment, scaling, and operational infrastructure for generative AI models hosted on Vertex AI, GKE (Google Kubernetes Engine), or hybrid inference servers.
• Implement or Understanding end-to-end LLM lifecycle pipelines: prompt management, fine-tuning workloads, automated retrieval-augmented generation (RAG) components, and evaluation harness pipelines.
• Deploy, manage, and scale vector databases (e.g., Vertex AI Vector Search, Pinecone, Qdrant, pgvector) with proper indexing, backup, and latency monitoring.
• Architect and optimize model-serving engines.

3. CI/CD & Pipeline Automation

• Build and maintain scalable CI/CD pipelines using Cloud Build, GitHub Actions, or GitLab CI for both microservices and machine learning models.
• Establish automated testing for infrastructure changes (e.g., tflint, tfsec, terratest) and model release pipelines (canary deployments, blue/green rollouts, shadow traffic).
• Package and deploy containerized AI/ML applications using Docker, Artifact Registry, and Helm charts on GKE.

4. Observability, Guardrails & Reliability

• Set up deep observability stacks using Google Cloud Monitoring, OpenTelemetry, Prometheus, and Grafana to track standard SRE metrics (SLIs/SLOs, latency, throughput).
• Implement LLM-specific telemetry: token consumption rates, inference latency, model drift, cost-per-query, and hallucinations/groundedness metrics using tools such as LangSmith, Phoenix/Arize, or Vertex AI Model Monitoring.
• Integrate safety guardrails, prompt sanitization, rate limiting, and fallback/resiliency mechanisms across inference endpoints.

5. FinOps & Compute Optimization

• Monitor and optimize cloud spending for compute-intensive workloads, specifically LLM quota monitoring, spot instances, and dynamic autoscaling.
• Partner with engineering and product teams to project infrastructure costs and establish budget alerts and quota guardrails.

Requirements & Qualifications

Must-Have

• Experience: 5+ years in DevOps, SRE, or Cloud Platform Engineering with at least 3+ years dedicated to Google Cloud Platform (GCP).
• Terraform Expertise: Advanced proficiency with Terraform (state management, reusable module design, workspaces, policy-as-code with Sentinel or OPA).
• LLMOps / MLOps Background: 1.5+ years of practical experience operationalizing LLMs or ML workloads, including familiarization with Vertex AI (Model Garden, Pipelines, Endpoints) or custom model serving on Kubernetes.
• Containerization & Orchestration: Strong background in Docker and Google Kubernetes Engine (GKE), including cluster setup, GPU node pools, ingress controllers, and autoscaling (HPA, KEDA).
• CI/CD: Hands‑on experience configuring automated pipelines with GitOps patterns (e.g., ArgoCD, Flux) or modern CI engines (GitHub Actions, Cloud Build).
• Scripting & Languages: Solid programming/scripting proficiency in Python and Bash/Shell.
• Security & Networking: Strong understanding of GCP VPC architecture, private endpoints, service perimeters (VPC‑SC), and Secret Manager.

Nice-to-Have

• Experience deploying agentic frameworks (e.g., LangChain, LlamaIndex, LangGraph, CrewAI) or Model Context Protocol (MCP) servers in production.
• GCP Certifications (e.g., Google Cloud Certified Professional Cloud DevOps Engineer, Professional Cloud Architect, or Professional Machine Learning Engineer).
• Familiarity with distributed model training and high-throughput inference caching strategies (Redis, Semantic Cache).
• Exposure to regulatory compliance frameworks (SOC2, HIPAA, ISO 27001) in relation to cloud AI systems.