LLM Engineer - 2

Zoho; Pune District; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

Zoho is seeking a Senior LLM Engineer to lead the design, development, and deployment of autonomous agentic AI solutions in production environments. The role spans multi-agent coordination, tool calling, and enterprise workflow integration, with a strong emphasis on security, fairness, and reliability.

Applicants should have 4+ years of experience and deep Python expertise, along with hands-on work with LangGraph, LangChain, and vector databases.

Qualifications

• 1+ year experience with LangGraph is required.
• 4+ years of overall industry experience.
• Proficiency in LangChain and other agentic orchestration frameworks.
• Design and deploy multi-agent frameworks and sophisticated tool-calling pipelines.
• Experience with continuous monitoring of autonomous AI agents in production.
• Develop scalable LLM evaluation methodologies for safety and alignment.
• Proficiency in Python.
• Familiarity with RAG and modern vector databases (Pinecone, Weaviate).
• Experience with cloud platforms (AWS, GCP, Azure) and MLOps in production.

Responsibilities

• Architect, design, and deploy scalable agentic AI systems coordinating LLMs and external tools.
• Develop and optimize AI-driven pipelines using LangGraph, ADK, and LangChain.
• Integrate agentic AI into enterprise workflows automating complex business functions.
• Address system evaluation, debugging, and ensure regulatory and ethical compliance.
• Promote security, transparency, fairness, and reliability across deployments.
• Mentor junior engineers and drive adoption of state-of-the-art LLM practices.

Skills

LangGraph

LangChain

Python

Multi-agent frameworks

Tool calling / API orchestration

Cloud platforms

MLOps

Data privacy & compliance

Tools

Pinecone

Weaviate

Job description

The Senior LLM Engineer will lead the design, development, and deployment of complex, autonomous agentic AI solutions within production environments, working across multi-agent frameworks and orchestrating large language models. The position is or scalable applied AI, with a focus on delivering robust and fair agentic systems that deliver organizational value, autonomy, and operational efficiency.

Responsibilities

• Architect, design, and deploy scalable agentic AI systems that coordinate LLMs, multi-agent collaboration, and external tool integrations through advanced tool calling.
• Develop, maintain, and optimize AI-driven pipelines utilizing frameworks such as LangGraph, ADK and LangChain, enhancing data inference and orchestration.
• Integrate agentic AI solutions into enterprise workflows, automating complex business functions (e.g., customer service, supply chain, decision support).
• Proactively address system evaluation, debug AI agent workflows, and ensure compliance with emerging regulatory and ethical guidelines.
• Promote security, transparency, fairness, and reliability across all deployed agentic systems.
• Mentor junior engineers, contribute to cross-functional teams, and champion the adoption of state-of-the-art LLM technologies and practices.

Requirements

• At least 1 year of experience with LangGraph (must have)
• 4+ years of overall experience working on industry projects
• Experience with LangChain, and other agentic orchestration frameworks.
• Deep proficiency in designing and deploying multi-agent frameworks and sophisticated tool-calling or API orchestration pipelines.
• Design and implement continuous monitoring systems for autonomous AI agents, ensuring performance, reliability, adherence to ethical guardrails, and optimal cost/latency in production.
• Develop and execute scalable LLM evaluation methodologies (including code-based, LLM-as-a-Judge) to rigorously assess model quality, safety, and goal alignment.
• Proficiency in Python
• Familiarity with retrieval-augmented generation (RAG) and modern vector databases (Pinecone, Weaviate).
• Experience with cloud platforms (AWS, GCP, Azure), MLOps practices, and production-grade deployment.
• Strong understanding of data privacy, model security, and compliance with current AI regulations and frameworks.

Preferred Skills

• Experience with Hugging Face, Neo4j, or knowledge graphs.
• Background in autonomous decision-making systems, anomaly detection, and dynamic process optimization.
• Track record of publishing, presenting, or open-sourcing agentic AI innovations.
• Strong problem-solving aptitude, collaborative mindset, and excellent communication skills.