Senior AI Engineer Microcare India
Title - Senior AI Engineer
Location - Hyderabad, Telangana (on-site / hybrid; travel to customer sites in AP and Telangana)
Experience - 5 to 8 years in software engineering, with 3+ years building ML/GenAI systems in production
Employment - Full-time
Reports to Head, AI Practice
Team - AI Solutions & AI Labs practice
About Microcare
Microcare is a 37-year-old IT systems integrator headquartered in Hyderabad, with 350+
people and engineers across five locations in Andhra Pradesh and Telangana.
We have served 700+ customers and delivered 2,000+ projects.
The role
You will be a founding senior engineer in Microcare's AI practice, turning customer AI conversations into working, deployed systems.
Most of our customers need AI that runs on their own infrastructure on-premise GPU clusters we build on HPE, or private cloud tenancies so you will own solutions end to end, from model choice to production hand-
over.
What you will build
- Organization GPT framework a reusable, packaged platform we deploy as CampusGPT, BankGPT and other organization-specific assistants, on-prem or in cloud.
- Enterprise document search and RAG secure retrieval over large document estates, starting with our AI document search MVP for a regional rural bank AI
- AI lab enablement the software stack, sample workloads and experiment support for the GPU labs we install at engineering colleges and enterprises.
- Government and enterprise AI solutions agents, copilots and analytics delivered under empanelments and tenders.
Key responsibilities
- Design and build GenAI applications: RAG pipelines, conversational assistants, and tool-using agents on Claude, OpenAI, Azure OpenAI, AWS Bedrock and open-weight models (Llama, Mistral, Qwen).
- Own the core of the organization GPT framework ingestion, chunking and embeddings, vector search, prompt and model orchestration, role-based access, audit logs packaged for repeatable deployment.
- Deploy and serve models on on-prem NVIDIA GPU infrastructure (vLLM, TGI, Triton, Ollama) with Docker and Kubernetes; size GPU, memory and storage for customer workloads.
- Fine-tune and evaluate models (LoRA/QLoRA), including for Indian languages such as Telugu and Hindi where customers need it.
- Build evaluation and guardrails: retrieval quality, hallucination checks, red-teaming, PII handling, and data residency in line with Indias DPDP Act and sector rules (RBI for banks).
- Lead technical discovery with customers, write solution architectures and effort estimates, and support pre-sales on proposals and tenders.
- Mentor junior engineers and freshers, review code, and set engineering standards for the AI practice.
- Contribute to AI lab curricula and skill development programmes.
Must-have qualifications
- B.Tech/B.E., M.Tech or MCA in Computer Science or a related field.
- 6+ years of software engineering, with 3+ years building ML or GenAI systems that reached production.
- Strong Python; working knowledge of FastAPI or similar, SQL, and Git.
- Hands-on RAG experience: LangChain, LlamaIndex or equivalent; embeddings; vector databases (pgvector, Milvus, Qdrant, Chroma or Elasticsearch).
- Experience with at least one major LLM API (Claude, OpenAI/Azure OpenAI, Bedrock or Gemini) and with open-weight models.
- Experience deploying with Docker and Kubernetes on Linux, on-prem or in cloud.
- Document processing: PDF parsing, OCR, table extraction across messy, scanned enterprise documents.
- Clear communication in English with customer technical teams; able to explain trade-offs to non-technical stakeholders.
Nice to have
- GPU serving and performance tuning: CUDA basics, quantization (GPTQ, AWQ, GGUF), multi-GPU inference.
- Agent frameworks and protocols: Model Context Protocol (MCP), LangGraph, CrewAI, or the Claude Agent SDK.
- Indic language NLP and speech (e.g. AI4Bharat models, Bhashini).
- MLOps: MLflow, Kubeflow, model monitoring, CI/CD for ML.
- Experience with HPE, NVIDIA or other enterprise GPU hardware, or with private cloud (OpenShift, VMware).
- Delivery for banking, government or higher-education customers in India.
- Cloud AI certifications (Azure AI Engineer, AWS ML Specialty, NVIDIA DLI).
What we offer
- A senior seat in a new AI practice, with real influence over architecture and the products we take to market.
- Direct access to frontier models and partner programmes from Anthropic, OpenAI, Microsoft, AWS and HPE.
- Hands-on work with on-prem GPU infrastructure that most engineers only see in the cloud.
- Competitive compensation, health insurance, and a certification and learning budget.