AI Platform Engineer

Rubiscape; Pune District; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Rubiscape, based in Pune, India, seeks an AI Platform Engineer to own the infrastructure powering RubiAI and its agentic AI copilots. You will build foundations like the LLM gateway, RAG orchestration, prompt management, vector stores, and a real-time inference bus for enterprise SaaS, BYOC, and air-gap topologies.

This high-ownership role requires 4+ years in backend/Infra with AI/LLM in production, Python expertise, LangChain/LlamaIndex for RAG, and strong knowledge of vector databases.

Qualifications

• 4+ years in platform, infrastructure, or backend engineering with at least 2 years focused on AI/LLM systems in production.
• Deep hands-on experience with LangChain or LlamaIndex for RAG pipeline construction and agentic workflow orchestration.
• Proficiency in Python for high-performance backend services and familiarity with async frameworks (FastAPI, asyncio).
• Practical experience with vector databases (Weaviate, Qdrant, Pinecone, or pgvector) at production query volumes.
• Strong understanding of LLM deployment options: OpenAI/Anthropic API, Azure OpenAI, vLLM, Ollama, and quantised model inference.
• Experience designing multi-tenant AI services with security isolation, audit logging, and usage metering.

Responsibilities

• Design and operate Rubiscape’s LLM gateway: request routing, model failover, token budgeting, cost attribution, and rate-limit enforcement across multiple LLM providers and self-hosted models.
• Build and maintain the RAG pipeline infrastructure — document chunking, embedding generation, vector store management, and retrieval orchestration using LangChain or LlamaIndex.
• Own the prompt management system: versioning, A/B evaluation, regression testing, and deployment of prompt templates used across RubiAI and studio co-pilots.
• Architect low-latency inference APIs that serve both synchronous and streaming AI responses to the Rubiscape front-end and third-party integrations.
• Implement guardrails, content filtering, and PII redaction layers required for regulated-sector AI deployments.
• Work with the MLOps team to integrate fine-tuned and quantised open-source models into the platform’s model registry and serving infrastructure.
• Establish platform-wide observability: trace AI requests end-to-end using OpenTelemetry and internal dashboards.

Skills

Backend engineering

Distributed systems

High-performance backend

Tools

LangChain

LlamaIndex

Weaviate

Qdrant

Pinecone

pgvector

FastAPI

asyncio

OpenAI API

Azure OpenAI

Ollama

Job description

The AI Platform Engineer at Rubiscape isthe architect of the infrastructure layer that powers RubiAI — our agentic AIcopilot — and the shared AI services consumed by all six studios. You willbuild the foundations: LLM gateway, RAG orchestration layer, prompt managementsystem, vector store integrations, and the real-time inference bus that servesenterprise customers across SaaS, BYOC, and air-gap topologies. This is ahigh-ownership role at the core of Rubiscape’s differentiation as a unifiedDecision Intelligence Platform with 8 international innovation patents.

Key Responsibilities

• Design andoperate Rubiscape’s LLM gateway: request routing, model failover, tokenbudgeting, cost attribution, and rate-limit enforcement across multiple LLMproviders and self-hosted models.
• Build andmaintain the RAG pipeline infrastructure — document chunking, embeddinggeneration, vector store management (Weaviate/Qdrant/pgvector), and retrievalorchestration using LangChain or LlamaIndex.
• Own theprompt management system: versioning, A/B evaluation, regression testing, anddeployment of prompt templates used across RubiAI and studio co-pilots.
• Architectlow-latency inference APIs that serve both synchronous (sub-200ms) andstreaming (SSE/WebSocket) AI responses to the Rubiscape front-end andthird-party integrations.
• Implementguardrails, content filtering, and PII redaction layers required forregulated-sector AI deployments in BFSI, healthcare, and government.
• Work withthe MLOps team to integrate fine-tuned and quantised open-source models (Llama,Mistral, Phi) into the platform’s model registry and serving infrastructure.
• Establishplatform-wide observability: trace AI requests end-to-end from user querythrough retrieval, LLM call, and response synthesis using OpenTelemetry andinternal dashboards.

Nice to Have

• Experiencebuilding agentic AI systems with tool use, multi-step planning, and memorymanagement.
• Knowledgeof model compression techniques (quantisation, distillation, speculativedecoding) for on-premises or edge AI deployment.
• Exposureto enterprise AI governance frameworks and responsible AI principles relevantto regulated sectors.
• Contributionto open-source LLM or RAG projects.

About Rubiscape

Rubiscape is India’s leading DecisionIntelligence Platform, unifying data engineering, BI, machine learning, andagentic AI in a single governed platform. Built in Pune and trusted by Fortune500 enterprises across BFSI, manufacturing, healthcare, and government. 8international innovation patents. 10 Industry-Academia Labs & COEs. From BIto AI — One Platform. Every Decision.

Requirements

• 4+ yearsin platform, infrastructure, or backend engineering with at least 2 yearsfocused on AI/LLM systems in production.
• Deephands-on experience with LangChain or LlamaIndex for RAG pipeline constructionand agentic workflow orchestration.
• Proficiencyin Python for high-performance backend services and familiarity with asyncframeworks (FastAPI, asyncio).
• Practicalexperience with vector databases (Weaviate, Qdrant, Pinecone, or pgvector) atproduction query volumes.
• Strongunderstanding of LLM deployment options: OpenAI/Anthropic API, Azure OpenAI,vLLM, Ollama, and quantised model inference.
• Experiencedesigning multi-tenant AI services with security isolation, audit logging, andusage metering.