Sr. AI Engineer

Techmatic Systems India Pvt Ltd; Hyderabad; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Cybergateway in Hyderabad is seeking a Senior AI Engineer / AI Architect to design and build enterprise‑grade AI ecosystems powered by large language models and agentic architectures. You will architect AI agents, orchestration layers, retrieval systems, tool calling, and production‑level inference pipelines for scalable, secure deployments.

The ideal candidate has hands‑on experience with both cloud‑hosted foundation models and self‑hosted open‑source LLMs, strong emphasis on reliability,

Responsibilities

• Design enterprise AI architectures and production AI platforms.
• Build and deploy AI agents, orchestration layers, and multi‑agent systems.
• Develop AI systems using OpenAI, Anthropic Claude, Gemini, and self‑hosted LLMs.

Job description

Cybergateway, Hyderabad

Key Responsibilities

• Llama
• Design end-to-end AI architectures for enterprise-scale applications using Large Language Models and Agentic AI.
• Architect and build production-grade AI agents, autonomous workflows, and multi-agent systems capable of reasoning, planning, tool usage, and memory management.
• Develop AI systems using both commercial APIs (OpenAI, Anthropic Claude, Gemini) and locally hosted open-source models including Llama, Qwen, Mistral, DeepSeek, Gemma, Phi, and other foundation models.
• Build, fine-tune, optimize, and deploy local LLMs from scratch using modern inference frameworks and GPU-based deployments.
• Design hybrid AI architectures capable of intelligently routing requests between proprietary and local LLMs based on latency, cost, privacy, and task complexity.
• Architect scalable Retrieval-Augmented Generation (RAG) systems using vector databases, hybrid search, semantic retrieval, reranking, and metadata filtering.
• Design advanced tool-calling architectures integrating AI systems with enterprise databases, APIs, ERP systems, vector stores, and external services.
• Build production-ready Text-to-SQL pipelines with schema reasoning, SQL validation, deterministic execution, and secure query generation.
• Develop AI reliability mechanisms including structured outputs, response validation, guardrails, hallucination mitigation, grounding strategies, and semantic verification.
• Design scalable inference infrastructure supporting streaming responses, model serving, caching, batching, GPU optimization, and cost-efficient deployments.
• Build AI observability frameworks including tracing, evaluation, benchmarking, prompt versioning, response scoring, regression testing, and production monitoring.
• Develop secure multi-tenant AI systems with role-based access control, data isolation, prompt security, and enterprise-grade governance.
• Collaborate with engineering, product, and design teams to define AI architecture, technical direction, and best practices.
• Stay updated with advancements in LLM research, reasoning models, AI agents, orchestration frameworks, inference optimization, and open-source AI ecosystems.

Role Overview

We are looking for a highly skilled Senior AI Engineer / AI Architect with hands‑on experience in architecting and building enterprise‑grade AI systems powered by Large Language Models (LLMs), Agentic AI, and Multi‑Agent Architectures.

The ideal candidate should have deep expertise in designing scalable AI infrastructures using both cloud‑hosted foundation models (OpenAI, Anthropic Claude, Gemini, etc.) and self‑hosted/local open‑source LLMs (Llama, Qwen, Mistral, DeepSeek, Phi, Gemma, etc.).

You will be responsible for designing complete AI ecosystems including AI agents, orchestration layers, retrieval systems, tool calling, model routing, inference optimization, AI security, and production deployment.

This role goes far beyond chatbot development and requires experience in building reliable AI products from architecture through production deployment.

Job Details

• Role: Senior AI Engineer / AI Architect
• Experience Required: 5 to 8 Years
• Employment Type: Full-Time, Permanent
• Work Schedule: Monday to Friday | 5 Days Work From Office
• Work Location: Cybergateway, Hyderabad

Required Skills & Qualifications

• B.E. / B.Tech / M.Tech / M.Sc. in Computer Science, Artificial Intelligence, Machine Learning, Data Science, Information Technology, or related disciplines.

Core Technical Skills

AI Architecture

• Strong experience designing enterprise AI architectures and production AI platforms.
• Deep understanding of LLM architecture, transformer models, inference pipelines, embeddings, tokenization, and context management.

Large Language Models

Hands‑on experience with:

• OpenAI (GPT)
• Anthropic Claude
• Google Gemini
• Llama
• Qwen
• Mistral
• DeepSeek
• Gemma
• Phi
• Hugging Face ecosystem

Must have experience working with both API‑based models and self‑hosted/local open‑source LLMs.

Local Model Development

• Strong experience deploying, optimizing, and serving local LLMs from scratch.
• Experience with Ollama, vLLM, TensorRT-LLM, llama.cpp, Hugging Face Transformers, TGI, or similar inference frameworks.
• Experience with model quantization (GGUF, GPTQ, AWQ), LoRA/QLoRA fine‑tuning, GPU optimization, and distributed inference.
• Understanding of CUDA, NVIDIA GPU optimization, memory management, and high‑performance inference.

Agentic AI

Experience building AI agents using:

• LangGraph
• CrewAI
• AutoGen
• Semantic Kernel
• DSPy
• LangChain
• Custom Agent Frameworks

Strong understanding of:

• Multi‑agent systems
• Tool Calling
• Planning & Reasoning
• Memory Architecture
• Workflow Orchestration
• Autonomous AI Systems

Retrieval Systems

• RAG Architecture
• Hybrid Search
• Semantic Search
• Embeddings
• Vector Databases
• Metadata Filtering
• Context Engineering

Experience with:

• Pinecone
• Weaviate
• ChromaDB
• Milvus
• PGVector
• FAISS

AI Infrastructure

• FastAPI
• Python
• REST APIs
• WebSockets
• SSE Streaming
• PostgreSQL
• Redis
• Docker
• Kubernetes
• CI/CD
• AWS
• Azure
• GCP

AI Reliability

Experience implementing:

• Structured Outputs
• JSON Schema
• Pydantic
• Guardrails
• Hallucination Mitigation
• Grounding
• AI Evaluation
• Prompt Engineering
• AI Observability

Good to Have

• Experience fine‑tuning open‑source LLMs for domain‑specific use cases.
• Experience deploying AI infrastructure on on‑premise GPU clusters.
• Knowledge of Model Context Protocol (MCP) and AI tool ecosystems.
• Experience implementing semantic caching and intelligent model routing.
• Familiarity with distributed inference and scalable GPU serving.
• Experience with AI copilots, enterprise AI assistants, and conversational analytics platforms.
• Understanding of responsible AI, governance, explainability, and compliance.
• Contributions to open‑source AI projects or personal AI products.

What Makes You a Great Fit

We are specifically looking for engineers who have built real‑world AI platforms and architectures—not just AI chatbots.

Candidates with hands‑on experience in the following will be strongly preferred:

• Enterprise AI Architecture
• Agentic AI Systems
• Multi‑Agent Workflows
• Local LLM Development & Deployment
• OpenAI, Anthropic, Gemini & Open‑Source LLM Ecosystems
• AI Infrastructure & GPU Inference
• Text-to-SQL Systems
• Retrieval‑Augmented Generation (RAG)
• AI Reliability Engineering
• Production LLM Platforms
• AI Observability & Evaluation
• Secure Multi‑Tenant AI Systems

What We Offer

• The opportunity to work on impactful, production‑grade AI products used by real customers.
• A collaborative, innovation‑first work culture with access to the latest AI tools and research.
• Mentorship from senior technologists and exposure to end‑to‑end AI product development.
• Clear career growth path with opportunities to move into senior/lead engineering roles.
• A prime work location at Cybergateway, one of Hyderabad's leading tech hubs.
• A team that values your ideas, encourages experimentation, and celebrates learning from failure.