Principal AI Architect - Conversational AI & RAG Systems

Alphastream; Bengaluru; On-site; INR 2,500,000 - 3,500,000; Full time

Benefits offered by this job

Competitive compensation

Comprehensive benefits package

Opportunities for professional growth

Job summary

A financial technology company in India is seeking a Principal AI Architect to lead the design and architecture of its conversational AI platform. This role entails crafting end-to-end RAG pipelines and guiding a team of engineers. The ideal candidate should have over 8 years of experience in AI/ML engineering with a strong background in retrieving systems and technical leadership. The position offers competitive compensation and opportunities for career growth in a dynamic workplace.

Qualifications

• 8+ years in AI/ML engineering with production RAG or conversational AI systems experience.
• Proven track record architecting and deploying LLM-powered applications serving >10K users.
• Experience leading technical teams (4+ engineers) through architecture design.

Responsibilities

• Lead the design and architecture of conversational AI platforms.
• Architect end-to-end RAG pipelines and build tools for data orchestration.
• Guide implementation and partner with R&D leadership.

Skills

AI/ML engineering

Deep expertise in retrieval systems

Strong understanding of LLM architectures

Education

8+ years in AI/ML engineering

Tools

Python

Kubernetes

Docker

Job description

Principal AI Architect - Conversational AI & RAG Systems

8+ years

About the Role

We're looking for a Principal AI Architect to lead the design and architecture of our conversational AI platform that powers private credit analysis for institutional investors. You'll architect end-to-end RAG pipelines, build MCP servers over complex financial datasets, and design agentic workflows that enable multi-step reasoning across deals, funds, and market data.

This is a hands-on architectural role where you'll both design systems and guide implementation. You'll lead our Alpha AI squad, partner with R&D leadership on research direction, and set standards for AI/ML engineering across the company.

Impact: Your architecture will directly determine how fast and accurately our AI can analyse private credit portfolios ($100M+ AUM clients), which translates to deal-closing speed for asset managers.

What You'll Build

• Conversational AI Architecture
  • Design MCP (Model Context Protocol) server architecture for structured financial data access
  • Architect RAG pipeline: retrieval → re-ranking → context funneling → generation
  • Build tool orchestration layer: function calling, parameter validation, response parsing
  • Design agentic workflows with task decomposition and multi-turn reasoning
  • Example: Architect a system where "Compare portfolio risk across Q1 and Q2" triggers: data extraction → metric computation → trend analysis → response synthesis

Retrieval System Design

• Design hybrid search strategy combining keyword (BM25) + semantic (vector) search
• Implement re-ranking with cross-encoders and Maximum Marginal Relevance (MMR)
• Build citation system linking every generated statement to source document + page number
• Optimize query expansion, passage relevance scoring, and metadata filtering
• Performance target: Retrieval recall@10 > 90%, answer citation accuracy > 95%

MCP Server Development

• Build MCP servers over structured datasets: deal terms, fund performance, financials, market comps
• Design tool definitions with strict schemas: function signatures, parameter types, validation rules
• Implement guardrails: confidence thresholds, schema validation, human-in-the- loop triggers
• Example: MCP server for "Get deal covenants" must return structured JSON with covenant type, threshold, measurement period, breach conditions

System Observability & Quality

• Build observability framework: log queries, retrieval candidates, tool calls, responses, latencies
• Design RAG quality metrics: answer accuracy, citation precision, retrieval recall, hallucination rate
• Implement A/B testing framework for prompt strategies and retrieval configurations
• Target: <2s end-to-end latency (95th percentile), query success rate > 98%

Technical Leadership

• Lead Alpha AI squad (4-6 engineers): guide architectural decisions, review designs, unblock technical challenges
• Conduct design reviews for all AI/ML features across Analyst Platform and Insights Engine
• Mentor full-stack engineers on LLM integration patterns, prompt engineering, and AI best practices
• Partner with Head of R&D on research direction: what models to fine-tune, what benchmarks to chase

Must-Have Qualifications Experience & Expertise:

• 8+ years in AI/ML engineering with 3+ years designing production RAG or conversational AI systems
• Proven track record architecting and deploying LLM-powered applications serving >10K users or $10M+ revenue
• Deep expertise in retrieval systems: vector databases (Pinecone/Weaviate/Qdrant), embedding models, hybrid search, re-ranking
• Strong understanding of LLM architectures, fine-tuning, and prompt engineering (few-shot, CoT, ReAct patterns)
• Experience building agentic workflows with tool use, function calling, and multi- step reasoning

Technical Skills:

• Vector databases: Production experience with Vespa, Pinecone etc
• LLM frameworks: LangChain, LlamaIndex, or custom orchestration for complex workflows
• Embedding models: SentenceTransformers, OpenAI embeddings, domain- specific fine-tuning
• MCP or similar protocols: Experience building structured data access layers for LLMs
• Python (expert level): FastAPI, asyncio, Pydantic for API design and validation
• Evaluation frameworks: RAGAS, LangSmith, custom benchmarking for RAG quality

Leadership:

• Experience leading technical teams (4+ engineers) through architecture design and implementation
• Track record of setting technical standards and conducting design reviews across multiple products
• Ability to translate business requirements into technical architecture and success metrics

Added Advantage Domain & Scale:

• Experience in FinTech, credit analysis, or financial data platforms (understanding of deals, covenants, financials)
• Built systems processing >1M documents or 100GB+ knowledge bases
• Familiarity with private credit, structured finance, or asset management workflows
• Fine-tuned Small Language Models for domain-specific tasks (e.g., financial entity extraction, classification)
• Implemented query decomposition and planning
• Built multi-modal RAG systems (text + tables + charts)
• Experience with prompt optimization at scale (DSPy, automatic prompt tuning)

Infrastructure:

• Designed model serving infrastructure: inference optimization, batching, caching, A/B deployment
• Experience with Kubernetes, Docker, and ML Ops tooling (MLflow, Weights & Biases)

Research:

• Published papers or blog posts on RAG, retrieval, or agentic AI
• Contributed to open-source projects in LLM tooling or vector search

Who We Are

Alphastream.ai envisions a dynamic future for the financial world, where innovation is propelled by state-of-the-art AI technology and enriched by a profound understanding of credit and fixed- income research. Our mission is to empower asset managers, research firms, hedge funds, banks,

and investors with smarter, faster, and curated data. We provide accurate, timely information, analytics, and tools across simple to complex financial and non-financial data, enhancing decision-making. With a focus on bonds, loans,financials and sustainability, we offer near real- time data via APIs and PaaS (Platform as a Service) solutions that act as the bridge between our offerings and seamless workflow integration.

"At Alphastream.ai we offer a dynamic and inclusive workplace where your skills are valued and your career can flourish. Enjoy competitive compensation, a comprehensive benefits package, and opportunities for professional growth. Immerse yourself in an innovative work environment, maintain a healthy work-life balance, and contribute to a diverse and inclusive culture. Join us to work with cutting-edge technology, and be part of a team that recognizes and rewards your achievements, all while fostering a fun and engaging workplace culture."

Alphastream.ai is an equal opportunities employer. We work to provide a supportive and inclusive environment where all individuals can maximize their full potential. Our skilled and creative workforce is comprised of individuals drawn from a broad cross section of all communities in which we operate and who reflect a variety of backgrounds, talents, perspectives, and experiences. Our strong commitment to a culture of inclusion is evident through our constant focus on recruiting, developing, and advancing individuals based on their skills and talents.