Applied AI / LLM Specialist (Non-Developer)

Marvin; Delhi; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

A technology firm in Delhi is looking for an Applied AI Specialist with hands-on experience with Large Language Models (LLMs). In this full-time role, you will be responsible for designing LLM evaluation pipelines and collaborating with product teams to translate practical use cases into AI solutions. We're seeking someone who enjoys experimentation and has a solid grasp of AI problem-solving, ideally with knowledge of RAG systems. Competitive opportunities await for those eager to improve real-world LLM performance.

Qualifications

• Hands-on experience with GPT, Gemini, Claude, or similar LLMs is essential.
• Ability to design and implement LLM evaluation pipelines.
• Comfort with analyzing outputs and proposing enhancements.

Responsibilities

• Design and run LLM evaluation pipelines to assess quality and performance.
• Work with RAG systems including retrieval tuning and chunking strategies.
• Build agent-based workflows for practical applications.

Skills

Hands-on experience with LLMs

Experience building LLM evaluation frameworks

Knowledge of RAG and vector databases

Fine-tuning LLMs and ML training

Understanding of multi-step reasoning workflows

Experience with experimentation and analysis

Job description

Applied AI / LLM Specialist (Non-Developer)

We’re looking for an applied AI specialist with strong hands‑on experience working with Large Language Models (LLMs). This is a practical, implementation‑focused role, not a pure software engineering position.

What you’ll do

• Design and run LLM evaluation pipelines to assess quality, accuracy, and performance
• Work with RAG systems, including chunking strategies, embeddings, and retrieval tuning
• Build and iterate on agent‑based workflows for real‑world use cases
• Experiment with prompting, fine‑tuning, and model selection across LLM providers
• Analyze outputs, identify failure modes, and propose improvements
• Collaborate with product and engineering to translate use cases into AI solutions

What we’re looking for

• Strong hands‑on experience with LLMs (GPT, Gemini, Claude, etc.)
• Experience building and maintaining LLM evaluation frameworks
• Practical knowledge of RAG, vector databases, and retrieval strategies
• Experience with fine‑tuning LLMs and training ML models
• Solid understanding of agents, tools, and multi‑step reasoning workflows
• Comfortable working with experimentation, metrics, and qualitative analysis

Nice to have

• Experience with production AI systems
• Familiarity with AI observability and monitoring
• Background in applied ML or data science

This role is ideal for someone who enjoys applied AI problem‑solving, experimentation, and improving real‑world LLM performance rather than building large systems from scratch.

Seniority level

Not Applicable

Employment type

Full-time

Industries

Technology, Information and Internet