Gen AI LLMOps Engineer

Infosys; Bengaluru; On-site; INR 1,500,000 - 2,000,000; Full time

Job summary

Infosys is looking for a professional skilled in deploying and managing LLMs like OpenAI and Mistral in production environments. The role involves building scalable inference pipelines, integrating LLMs into applications, and implementing prompt engineering and retrieval-augmented generation (RAG).

The successful candidate will have expertise in managing LLM performance, and applying MLOps for version control and experiment tracking. Strong Python programming and experience with frameworks like LangChain are essential. Join us to operationalize Generative AI applications!

Qualifications

• Strong Python programming skills are essential.
• Hands-on experience with LLMs and Generative AI is required.
• Familiarity with LangChain, LangGraph, or LlamaIndex would be advantageous.

Responsibilities

• Deploy and manage LLMs in production environments.
• Build scalable inference pipelines for real-time and batch processing.
• Integrate LLMs into applications using APIs.
• Design end-to-end LLM pipelines including prompt engineering and RAG.
• Monitor LLM performance metrics for latency and accuracy.
• Implement MLOps for model updates and experiment tracking.

Skills

Strong Python programming

Hands-on experience with LLMs / Generative AI

Experience with LangChain / LangGraph / LlamaIndex

Solid understanding of RAG architecture

Experience building APIs using FastAPI / Flask

Job description

Qualifications

• Strong Python programming
• Hands‑on experience with LLMs / Generative AI
• Experience with LangChain / LangGraph / LlamaIndex
• Solid understanding of RAG architecture, prompt engineering, embeddings & vector search
• Experience building APIs using FastAPI / Flask

Responsibilities

Deploy and manage LLMs (OpenAI, Llama, Mistral, etc.) in production environments.

Build scalable inference pipelines (real‑time & batch).

Integrate LLMs into applications via APIs and microservices.

Design and implement end‑to‑end LLM pipelines: prompt engineering, retrieval‑augmented generation (RAG), fine‑tuning / embeddings, using frameworks such as LangChain, LangGraph, LlamaIndex.

Build and optimize RAG pipelines using vector databases (Pinecone, FAISS, Weaviate, Chroma), handle document ingestion, chunking, indexing, and retrieval.

Monitor LLM performance: latency, accuracy / hallucinations, cost efficiency.

Implement prompt optimization, feedback loops, guardrails & evaluation frameworks.

MLOps for LLMs: build CI/CD pipelines for model updates, prompt/version control, experiment tracking, deployments; ensure reproducibility of LLM workflows.

What This Role Is NOT

• Pure data scientist (model building only)
• Platform engineer (infra‑heavy role)
• Traditional MLOps without LLM exposure

This role focuses on

• LLM deployment + RAG + GenAI pipelines
• Operationalizing GenAI applications