AI / ML Ops Engineer

Zoho; Bengaluru South; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

Zoho is seeking an experienced AI / ML Ops Engineer to bridge data science and cloud infra. Build automated pipelines to train, test, deploy, and monitor ML models and generative AI systems securely at scale within enterprise container environments.

Responsibilities include orchestrating containerized model deployments on Kubernetes, implementing robust model tracking with Feast and DVC, and optimizing inference with ONNX/TensorRT while ensuring secure access controls and cost efficiency.

Qualifications

• 3+ years building and maintaining MLOps automation infrastructures.
• Strong Python, Docker, Kubernetes, ML frameworks, and SQL mastery.
• Deep understanding of distributed systems and GPU resource management.
• Experience with RAG pipelines or fine-tuning LLMs in production.
• Familiarity with Terraform for provisioning ML clusters.

Responsibilities

• Orchestrate containerized model deployments on Kubernetes.
• Set up low-latency inference endpoints and auto-scaling.
• Implement model tracking and data versioning with Feast and DVC.
• Build monitoring for model accuracy, data drift, and latency.
• Optimize inference with ONNX/TensorRT and quantization.
• Implement LLM Ops, vector DB scaling, and semantic caching.
• Enforce ML access controls and encryption for logs.

Skills

Python

Docker

Kubernetes

PyTorch

TensorFlow

Hugging Face

SQL

RAG pipelines

CI/CD

Tools

KServe

Triton Inference Server

Feast

DVC

ONNX

TensorRT

Pinecone

Milvus

Terraform

Job description

• Relevant Experience Required: 3+ years of dedicated MLOps or DevOps experience deploying and managing machine learning models in production
  
• Mandatory Certification: AWS Certified Machine Learning - Specialty, Google Cloud Certified Professional Machine Learning Engineer, or Databricks Certified Machine Learning Professional
  

Job Summary

We are seeking an experienced AI / ML Ops Engineer to bridge the gap between data science research and cloud infrastructure execution. The ideal candidate will build automated pipelines to train, test, deploy, and monitor machine learning models and generative AI systems securely and at scale within enterprise container environments.

• Orchestrate containerized model deployments, configuring low-latency inference endpoints, auto-scaling GPU/CPU clusters, and model serving runtimes on Kubernetes (KServe, Triton Inference Server).
  
• Implement robust model tracking and data versioning foundations, managing feature stores (e.g., Feast), model registries, and version controls for massive datasets using DVC.
  
• Build automated AI performance and data monitoring gates, tracking model accuracy decay, data drift indicators, concept drift parameters, and system processing latencies in real time.
  
• Optimize inference execution environments , leveraging model compilation engines (e.g., ONNX , TensorRT ) and quantization strategies to shorten response times and minimize cloud compute costs.
  
• Integrate generative AI and LLM operational frameworks (LLMOps), configuring semantic caching layers, vector database scaling parameters (e.g., Pinecone, Milvus), and prompt validation pipelines.
  
• Govern machine learning access controls and security profiles, configuring strict data separation barriers, model access tokens, and encryption protocols to safeguard sensitive inference logs.
  

Requirements

• 4 to 8 years of core software engineering, DevOps, or data engineering experience, with 3+ dedicated years actively building and maintaining MLOps automation infrastructures.
  
• Strong technical mastery of Python programming, container orchestration (Docker, Kubernetes), ML frameworks (PyTorch, TensorFlow, Hugging Face), and advanced SQL.
  
• Deep structural understanding of distributed system mechanics, GPU resource management limits, model deployment patterns (Shadow, Canary, A/B), and cloud provider API governance.
  
• Prior experience implementing RAG (Retrieval-Augmented Generation) pipelines or fine-tuning open-source LLM layers in production.
  
• Familiarity with infrastructure-as-code scripting tools like Terraform to provision ML cluster topologies.