GEN AI Architect

Calsoft; Pune District; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Calsoft seeks a highly experienced Generative AI Architect to design, develop, and deploy scalable GenAI solutions. You will lead end-to-end AI architectures, build RAG pipelines, and create multi‑tool agent systems with LangChain and LangGraph.

You will mentor engineers, define best practices, and engage with presales and RFP activities while advancing model evaluation and production observability using LangSmith, Phoenix, or Langfuse. Proficiency in Python and ML frameworks is required.

Qualifications

• BE/BTech with 10-15 years of progressive experience in software engineering, data science, or ML engineering roles.
• Minimum of 5 years of dedicated experience in developing and deploying Artificial Intelligence and Machine Learning solutions.

Responsibilities

• Design and lead the implementation of end‑to‑end, high‑performance Generative AI architectures, focusing on scalability, cost‑efficiency, and resilience.
• Architect and optimize Retrieval‑Augmented Generation (RAG) pipelines for enterprise‑specific knowledge bases, including vector store integration, indexing, and retrieval optimization.
• Design and implement complex, multi‑step Agentic AI systems and sophisticated conversational flows using frameworks like LangChain and LangGraph.
• Establish robust MLOps and GenAI‑Ops practices, integrating observability and evaluation tools such as LangSmith, Phoenix, or Langfuse to monitor model performance, latency, cost, and drift in production.
• Define and implement LLM evaluation methodologies, utilizing tools like Ragas to quantitatively assess the quality (e.g., faithfulness, answer relevance, context adherence) of RAG and Agentic AI applications.
• Continuously evaluate new LLMs, foundation models, open‑source frameworks, and emerging GenAI technologies to recommend and integrate the best‑fit solutions.
• Work closely with Data Scientists, ML Engineers, and business stakeholders to translate complex business problems into technical AI/ML roadmaps and provide technical guidance to development teams.
• Leverage a strong background in ML/NLP to advise on and implement techniques for prompt engineering, model fine‑tuning (e.g., LoRA, QLoRA), and advanced text processing.
• Be part of Technical Presales, Proposals and RFP response teams.

Skills

Gen AI Development

Machine Learning

Retrieval Augmented Generation (RAG)

Langchain

Vector Databases

Langgraph

Arize Phoenix

RAGAs

LLM-as-a-Judge

MCP

Python

PyTorch

TensorFlow

Hugging Face

AWS

Azure

GCP

Kubernetes

Docker

Education

BE/BTech

Tools

LangSmith

Phoenix

Langfuse

Ragas

Job description

• Position – Generative AI Architect


• Experience – 10+ Years


• Location – Pune/ Bangalore/ Kolkata/ Indore


• Skills – Gen AI Development Machine Learning, Retrieval Augmented Generation (RAG), Langchain, Vector Databases, Langgraph, Arize Phoenix, RAGAs, LLM-as-a-Judge, MCP

Notice Period – Immediate to 15 Days

Job Description

Overview We are seeking an exceptionally skilled and experienced Generative AI Architect to lead the design, development, and deployment of cutting-edge Generative AI solutions. This role requires a blend of deep machine learning and natural language processing expertise with hands‑on architectural experience in building scalable, production‑grade LLM applications. The ideal candidate will be a technical leader who can drive innovation, define best practices, and mentor engineering teams in the evolving GenAI space.

Key Responsibilities

• Design and lead the implementation of end‑to‑end, high‑performance Generative AI architectures, focusing on scalability, cost‑efficiency, and resilience.


• Architect and optimize Retrieval‑Augmented Generation (RAG) pipelines for enterprise‑specific knowledge bases, including vector store integration, indexing, and retrieval optimization.


• Design and implement complex, multi‑step Agentic AI systems and sophisticated conversational flows using frameworks like LangChain and LangGraph.


• Establish robust MLOps and GenAI‑Ops practices, integrating observability and evaluation tools such as LangSmith, Phoenix, or Langfuse to monitor model performance, latency, cost, and drift in production.


• Define and implement LLM evaluation methodologies, utilizing tools like Ragas to quantitatively assess the quality (e.g., faithfulness, answer relevance, context adherence) of RAG and Agentic AI applications.


• Continuously evaluate new LLMs, foundation models, open‑source frameworks, and emerging GenAI technologies to recommend and integrate the best‑fit solutions.


• Work closely with Data Scientists, ML Engineers, and business stakeholders to translate complex business problems into technical AI/ML roadmaps and provide technical guidance to development teams.


• Leverage a strong background in ML/NLP to advise on and implement techniques for prompt engineering, model fine‑tuning (e.g., LoRA, QLoRA), and advanced text processing.


• Be part of Technical Presales, Proposals and RFP response teams.

Qualifications

• BE/BTech with 10-15 years of progressive experience in software engineering, data science, or ML engineering roles. Minimum of 5 years of dedicated experience in developing and deploying Artificial Intelligence and Machine Learning solutions.

Mandatory Technical Skills

• Proven experience architecting and shipping production‑grade GenAI applications. Deep, hands‑on expertise in designing and optimizing Retrieval‑Augmented Generation (RAG) systems.


• Expert‑level proficiency with LangChain and LangGraph for building complex LLM chains, tools, and stateful agent workflows.


• Demonstrated experience in building and deploying multi‑tool, reasoning‑based Agentic AI systems.


• Hands‑on experience with GenAI‑specific observability and evaluation platforms like LangSmith, Phoenix, or Langfuse.


• Practical knowledge and experience using LLM evaluation frameworks, specifically Ragas, for quality assessment.


• High proficiency in Python and relevant data science libraries (e.g., PyTorch, TensorFlow, Hugging Face).


• Experience with cloud platforms (AWS, Azure, or GCP) and MLOps tools (e.g., Kubernetes, Docker) for scalable model deployment.

Soft Skills

• Excellent written and verbal communication skills, with the ability to clearly articulate complex technical concepts to both technical and non‑technical audiences.


• Demonstrated ability to lead technical initiatives, influence architectural decisions, and mentor junior and senior engineers.