LLM Integration / LangChain Engineer

Zoho; India; Remote; INR 2,500,000 - 4,200,000; Full time

Job summary

Zoho is seeking an experienced LLM Integration Engineer to design and implement orchestration layers that connect enterprise data assets with generative AI models.

You will build retrieval-augmented generation pipelines, develop multi-agent reasoning loops, and ensure secure middleware for AI at scale. Strong Python/TypeScript, LangChain, and cloud certification are required.

Qualifications

• 2+ years of production-level LLM integration experience.
• 2+ years actively writing production-level code around LLM infrastructures.
• Strong understanding of Python/TypeScript, vector representations, and prompt engineering.
• Mandatory cloud certification from AWS/GCP/Azure.

Responsibilities

• Design and construct advanced LLM applications and orchestrations using LangChain/LangGraph/LlamaIndex/AutoGen.
• Build production-grade RAG architectures with dynamic context handling and document parsing.
• Develop multi-agent reasoning chains, tool calls, and memory caching workflows.
• Expose and consume programmatic endpoints via high-throughput API integrations with databases/CRMs.
• Monitor token usage, latency, and prompt drift using observability tools.
• Implement secure middleware barriers for data sanitization and prompt safety.

Skills

Python

TypeScript

Vector representations

Prompt engineering

FastAPI

SQL

Transformer models

API orchestration

Education

Cloud certification (AWS/GCP/Azure)

Tools

LangChain

LangGraph

LlamaIndex

AutoGen

GPTCache

Job description

• Relevant Experience Required: 2+ years of dedicated hands-on experience building, integration, and deploying applications powered by Large Language Models (LLMs)
  

We are seeking an experienced LLM Integration / LangChain Engineer to design, develop, and implement the orchestration layers connecting our enterprise data assets with cutting-edge generative AI models. The ideal candidate will build production-grade Retrieval-Augmented Generation (RAG) pipelines, program multi-agent reasoning loops using LangChain, LangGraph, or LlamaIndex , and establish secure system middleware to safely deploy AI capabilities at scale.

Key Responsibilities

• Design and construct advanced LLM applications and orchestrations using specialized frameworks like LangChain , LangGraph , LlamaIndex , or AutoGen .
  
• Build production-grade Retrieval-Augmented Generation (RAG) architectures, configuring dynamic context chunking, document parsing, semantic metadata tagging, and reranking pipelines.
  
• Develop complex multi-agent reasoning chains and workflows, implementing custom tool calling structures, memory caching architectures, and guardrail validations.
  
• Expose and consume programmatic endpoints , constructing high-throughput API integrations connecting foundational LLMs (e.g., OpenAI , Anthropic , open-source models via Hugging Face/Ollama ) with internal corporate databases and CRMs.
  
• Apply rigorous AI evaluation and prompt tracking structures , utilizing observability platforms (e.g., LangSmith , Arize Phoenix ) to monitor token usage bounds, model latency, and prompt generation drift.
  
• Implement secure middleware execution barriers, configuring text sanitization, PII data-masking pipelines, prompt injection defensive rings, and toxicity filtering parameters.
  
• Optimize model inference costs and context window budgets , designing custom semantic caching frameworks (e.g., GPTCache ) to intercept recurring operational queries.
  

Requirements

• 4 to 8 years of core enterprise backend web engineering or data pipelines experience, with 2+ dedicated years actively writing production-level application code wrapped directly around LLM infrastructures.
  
• Strong technical mastery of Python or TypeScript, vector representations, prompt engineering grounding mechanics, asynchronous web frameworks (FastAPI), and SQL.
  
• Deep structural understanding of transformer model designs, text embedding properties, agentic tool execution cycles, and API orchestration limits.
  
• Mandatory certification: Professional-level machine learning or cloud developer certification from a major cloud vendor (AWS/GCP/Azure).
  

Preferred Qualifications

• Familiarity with deploying AI applications within container systems (Docker, Kubernetes) integrated into modern DevSecOps CI/CD delivery loops.