AI Software Architect (LLM & Agentic Systems)

Aziro; Hyderabad; Hybrid; INR 3,000,000 - 6,000,000; Full time

Job summary

Aziro in Hyderabad seeks an AI Software Architect/Senior AI Engineer to design and deliver production-grade AI solutions powered by Large Language Models. You will own AI service architecture, drive architectural decisions, and collaborate across backend, data, and cloud teams to scale RAG-based systems.

The role focuses on building agentic workflows, evaluating AI quality, ensuring observability, and deploying on cloud platforms with a strong emphasis on scalability, reliability, and cost

Qualifications

• Master's degree in Computer Science, AI, ML, or related field.

Responsibilities

• Own the architecture of AI-powered services and their integration with backend, mobile, and web applications.
• Design, build, and maintain production-grade LLM applications using modern AI frameworks and orchestration platforms.
• Lead technical design reviews and drive architectural decisions across AI services, backend systems, data pipelines, and cloud infrastructure.
• Architect agentic workflows involving tool use, function calling, multi-agent systems, planning, memory management, and reasoning chains.
• Establish evaluation frameworks to measure AI quality, reliability, latency, cost, and business impact.
• Implement AI observability and monitoring using platforms such as Langfuse, LangSmith, OpenTelemetry, and related tooling.
• Design and implement Retrieval-Augmented Generation (RAG) architectures using vector databases, embeddings, document pipelines, and retrieval optimization techniques.
• Collaborate with software engineers, data scientists, product managers, and domain experts to translate business requirements into AI solutions.
• Optimize prompts, retrieval strategies, model selection, and system architecture for accuracy, reliability, performance, and cost efficiency.
• Design scalable APIs and services to expose AI capabilities across internal and external applications.
• Define AI engineering standards, best practices, and governance processes across the organization.
• Provide technical leadership and mentorship to engineers working on AI initiatives.
• Leverage cloud platforms such as Google Cloud Platform (GCP) to deploy and scale AI services.
• Document AI architectures, workflows, evaluation methodologies, and operational procedures.

Skills

Python development

LLM architectures

Prompt engineering

Distributed systems

Cloud platforms

Education

Master's degree in Computer Science or related field

Tools

LangGraph

LangChain

LlamaIndex

CrewAI

Langfuse

LangSmith

Job description

We are looking for an AI Software Architect / Senior AI Engineer who is a self-starter and thrives on designing and delivering production-grade AI solutions powered by Large Language Models (LLMs). This role requires deep expertise in architecting, building, and operating complex AI systems, including Retrieval-Augmented Generation (RAG), agentic workflows, tool calling, evaluation frameworks, and observability platforms.

The ideal candidate combines strong software engineering fundamentals with demonstrated experience delivering real-world AI products. Beyond experimentation, this individual must be capable of designing scalable, reliable, and cost-effective LLM-powered solutions that operate successfully in production environments. They should possess a strong understanding of modern AI architecture patterns, prompt engineering, retrieval systems, agent orchestration, and AI observability.

Key Responsibilities

• Own the architecture of AI-powered services and their integration with backend, mobile, and web applications.
• Design, build, and maintain production-grade LLM applications using modern AI frameworks and orchestration platforms.
• Lead technical design reviews and drive architectural decisions across AI services, backend systems, data pipelines, and cloud infrastructure.
• Architect agentic workflows involving tool use, function calling, multi-agent systems, planning, memory management, and reasoning chains.
• Establish evaluation frameworks to measure AI quality, reliability, latency, cost, and business impact.
• Implement AI observability and monitoring using platforms such as Langfuse, LangSmith, OpenTelemetry, and related tooling.
• Design and implement Retrieval-Augmented Generation (RAG) architectures using vector databases, embeddings, document pipelines, and retrieval optimization techniques.
• Collaborate with software engineers, data scientists, product managers, and domain experts to translate business requirements into AI solutions.
• Optimize prompts, retrieval strategies, model selection, and system architecture for accuracy, reliability, performance, and cost efficiency.
• Design scalable APIs and services to expose AI capabilities across internal and external applications.
• Define AI engineering standards, best practices, and governance processes across the organization.
• Provide technical leadership and mentorship to engineers working on AI initiatives.
• Leverage cloud platforms such as Google Cloud Platform (GCP) to deploy and scale AI services.
• Document AI architectures, workflows, evaluation methodologies, and operational procedures.

Qualifications

• Master's degree in Computer Science, Artificial Intelligence, Machine Learning, Software Engineering, or related field.
• 10+ years of software engineering experience with at least 5 years focused on LLM-based solutions and generative AI systems.
• Demonstrated experience designing and deploying complex production-grade AI applications using Large Language Models.
• Extensive experience with AI orchestration frameworks such as LangGraph, LangChain, LlamaIndex, CrewAI, or equivalent technologies.
• Hands-on experience implementing AI observability and evaluation frameworks using Langfuse, LangSmith, or similar platforms.
• Proven expertise with Retrieval-Augmented Generation (RAG) architectures and vector database technologies.
• Strong understanding of modern LLM architectures, prompting strategies, context management, embeddings, and retrieval techniques.
• Strong Python development experience and software engineering fundamentals.
• Experience designing scalable APIs and cloud-native architectures.
• Ability to evaluate architectural trade-offs involving model performance, latency, reliability, maintainability, and cost.
• Experience deploying AI solutions in production environments using Docker, Kubernetes, and cloud platforms.
• Strong understanding of structured and unstructured data processing pipelines.
• Familiarity with modern database technologies including PostgreSQL, vector databases, and document stores.
• Excellent communication, leadership, and mentoring skills and ability to collaborate effectively with cross-functional teams.

Preferred Skills

• Experience building agentic systems involving tool use, planning, memory, and multi-agent orchestration and skills.
• Experience with model evaluation, benchmarking, AI testing frameworks, and automated quality assessment.
• Experience working with multiple commercial and open-source models including OpenAI, Anthropic, Gemini, Llama, and Mistral.
• Familiarity with fine-tuning, synthetic data generation, and model optimization techniques.
• Familiarity with developing and training machine learning models.
• Experience supporting AI products in regulated, privacy-sensitive, or high-availability environments.
• Experience integrating AI capabilities into mobile and web applications.
• Familiarity with modern software delivery practices including DevOps, CI/CD, and Agile development methodologies.