S&C GN - TS&T - Cloud AI Infra Architect - Manager

Accenture; Bengaluru; On-site; INR 4,500,000 - 7,500,000; Full time

Job summary

Accenture is seeking an AI Infrastructure Architect - Manager to design enterprise AI infrastructure across compute, networking, and governance. This design role focuses on standards, reference architectures, and governance frameworks for engineering teams.

The candidate will advise clients on AI infrastructure decisions, balancing cost, performance, and security. The role emphasizes infrastructure architecture rather than model development, with a global network focus and multiple delivery

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or closely related technical discipline. A Masters preferred.

Responsibilities

• Design end-to-end AI infrastructure architecture for enterprise clients across compute, networking, and governance.
• Serve as trusted technical advisor on AI infrastructure decisions, balancing cost, performance and security.
• Define standards, reference architectures, and governance for engineering teams.
• Focus on AI infrastructure, platform engineering, governance, and architecture, not model development.

Skills

AI infrastructure

Kubernetes

Terraform

MLOps

Vector databases

Cloud platforms

Security governance

Education

Bachelor's Degree in CS/IT/Engineering

Tools

AKS/EKS/GKE

Job description

Accenture is a leading global professional services company, providing a broad range of services and solutions in strategy, consulting, digital, technology and operations. Combining unmatched experience and specialized skills across more than 40 industries and all business functions - underpinned by the worlds largest delivery network - Accenture works at the intersection of business and technology to help clients improve their performance and create sustainable value for their stakeholders. With 750K people serving clients in more than 120 countries, Accenture drives innovation to improve the way the world works and lives.

About Global Network

Accenture Strategy shapes our clientsfuture, combining deep business insight with the understanding of how technology will impact industry and business models. Our focus on issues such as digital disruption, redefining competitiveness, operating and business models as well as the workforce of the future helps our clients find future value and growth in a digital world. Today, every business is a digital business. Digital is changing the way organisations engage with their employees, business partners, customers, and communities - how they manufacture and deliver products and services, and how they run their organisations. This is our unique differentiator. We seek people who recognise and understand the impact that digital and technology have on every industry and every sector, and share our passion to shape unique strategies that allow our clients to succeed in this environment.

To bring this global perspective to our clients, Accenture Strategys services include those provided by our Global Network - a distributed management consulting organisation that provides management consulting and strategy expertise across the client lifecycle. Approximately 10,000 consultants are part of this rapidly expanding network, providing specialised and strategic industry and functional consulting expertise from key locations around the world. Our Global Network teams complement our in-country teams to deliver cutting-edge expertise and measurable value to clients all around the world. For more information visit www.accenture.com/capabilitynetwork

Global Network Videos

Video Title

External Link

Accenture Global Network

Accenture in One Word

MBA Careers: What makes Accenture different

Accenture Inclusion The Power of Diversity

AI Infrastructure Architect - Manager

Practice Overview

Skill / Operating Group

Technology Consulting - AI Infrastructure Advisory

Level

Manager

Location

Gurugram / Mumbai / Bangalore / Pune / Kolkata

Travel

Expected travel could be anywhere between 0100%

Why Technology Consulting

The Technology Consulting business within Global Network invents the future for clients by providing them with the right guidance, design thinking and innovative solutions for technological transformation. Specialise in AI infrastructure consulting to transform the worlds leading organisations by: Helping Clients rethink their AI compute, networking, retrieval, and agent orchestration layers to scale enterprise AI reliably. Enhancing your Skillset with hyperscaler AI infrastructure (AWS, Azure, GCP), Kubernetes, Terraform, MLOps/LLMOps tooling, and agentic AI architecture. Transforming Businesses by designing next-generation AI infrastructure foundations that move clients from AI pilots to enterprise-scale production deployments.

Principal Duties Responsibilities

Overview

Design and deliver end-to-end AI infrastructure architecture for enterprise clients - setting the architecture vision for compute, networking, retrieval, agent orchestration, and the tooling layers that underpin every AI and agentic workload.

Serve as the trusted technical advisor to client architects and technology leads on AI infrastructure decisions, translating architectural choices into clear trade-offs across cost, performance, scalability, and security.

This is a design and architecture role, not a hands-on build role. The mandate is to define standards, reference architectures, and governance frameworks that engineering teams implement.

This role focuses on AI infrastructure, platform engineering, governance, and architecture. It does not focus on model development, prompt engineering, model fine-tuning, or AI application development. Applications from AI Engineers, Prompt Engineers, or ML practitioners without infrastructure architecture experience are unlikely to be a strong fit.

Key Responsibilities

1. AI Compute Infrastructure Architecture

• Design cloud AI infrastructure for enterprise workloads - GPU/CPU compute, Kubernetes (AKS, EKS, GKE), managed ML compute, networking, identity, and security across AWS, Azure, or GCP.
• Define reference architecture for AI Center of Excellence infrastructure - model hosting, agent environments, and the compute and networking beneath them.
• Architect compute strategy - GPU SKU selection, GPU economics and capacity planning, provisioned vs pay-as-you-go inference, and shared vs dedicated capacity models.
• Design AI-specific landing zones with network isolation, private connectivity for AI services, and AI guardrails at the workload boundary.

2. AI Platform Architecture

• Design enterprise AI platform architectures using Azure AI Foundry, AWS Bedrock, and GCP Vertex AI.
• Define model catalog governance, model lifecycle management, platform operating models, and AI self-service enablement patterns.
• Architect integration patterns between Azure AI Foundry, AWS Bedrock, and GCP Vertex AI and the underlying Kubernetes (AKS / EKS / GKE) compute and networking infrastructure.

3. Inference Layer Architecture

• Architect model serving strategy across managed cloud endpoints and self-hosted inference (e.g. vLLM/Triton on GPU-backed Kubernetes) - right pattern per use case.
• Design inference routing - model fallback chains, multi-model routing, and load balancing across provisioned and on-demand deployments.
• Define latency and throughput architecture for real-time and batch inference, including token-level latency budgets and streaming response design.
• Architect inference-layer caching - prompt caching and semantic caching to reduce redundant model calls and control token economics and cost.

4. Retrieval / RAG Architecture

• Design vector database architecture for RAG - select and size across cloud AI search, vector-enabled databases, and dedicated vector stores based on scale and latency needs.
• Architect embedding pipeline infrastructure - embedding model selection, reindexing strategy, and compute/storage patterns for chunking at enterprise scale.
• Define hybrid search architecture (vector, keyword, metadata filtering) and the infrastructure to support it at scale.
• Architect multi-tenant vector store isolation - ensuring one business units embedded data cannot leak into anothers retrieval results through index- and access-level boundaries.
• Design knowledge graph and GraphRAG infrastructure - graph database hosting and its integration pattern with the vector retrieval layer.

5. Agentic Tool Calling Architecture

• Architect tool calling infrastructure - how agents securely discover, authenticate to, and invoke internal APIs, MCP servers, and third-party connectors.
• Design MCP server hosting and governance - versioning, security, and reuse across agents and business units.
• Define the integration security boundary - scoped credentials, rate limiting per agent-to-tool call, and audit logging of every tool invocation.
• Architect agent runtime environments for agent orchestration frameworks (LangGraph, Semantic Kernel, CrewAI) on Kubernetes or serverless containers.
• Design multi-agent orchestration infrastructure, including agent-to-agent communication patterns and short-term/long-term agent memory storage.

6. Identity, Access AI Governance Architecture

• Define identity architecture for AI workloads - Managed Identity strategy for model access and RBAC for AI resource governance.
• Design private connectivity for AI services - ensuring inference traffic does not traverse the public internet.
• Architect network segmentation between inference, training, and agent orchestration workloads.
• Design infrastructure-level Responsible AI controls and AI guardrails - content filtering integration points, prompt-injection mitigation, and data residency controls.
• Define model governance standards - model registry architecture, version control, promotion gates, and audit trails for deployed models and agents.

7. Resilience, Scalability Cost Optimisation

• Design multi-region resilience patterns for AI workloads - endpoint failover, model endpoint high availability, and project replication.
• Architect autoscaling for inference workloads, accounting for GPU cost sensitivity and cold-start latency.
• Define capacity and quota management across subscriptions for shared AI services.
• Apply Well-Architected cost optimisation principles to AI infrastructure - rightsizing compute, reserved capacity planning, GPU utilisation efficiency, and token-level consumption awareness.

8. AI Observability, Governance MLOps Architecture

• Define the technical governance framework - model registry architecture, endpoint lifecycle management, and model governance standards for deployed models and agents.
• Architect the MLOps/LLMOps foundation - CI/CD for model and agent deployment using Terraform and native hyperscaler tooling, prompt versioning, and evaluation pipelines.
• Design AI observability architecture - token usage, per-inference latency, model drift signals, GPU utilisation monitoring, and hallucination-tracking hooks.
• Define enterprise standards for how agents are hosted, secured, and connected to systems.
• Provide architecture design review for AI use cases proposed by business units, ensuring infrastructure is scalable, reusable, and aligned to enterprise standards.
• Track hyperscaler AI roadmaps (AWS, Azure, GCP) and proactively incorporate emerging capabilities into the reference architecture.

Practice Building

• Codify methods and frameworks into reusable AI infrastructure assets for use across engagements.
• Create differentiated infrastructure offerings, accelerators, and reference implementations.
• Mentor team members in AI infrastructure architecture.
• Support engagement budgets, forecasting, and financial proposals.

Qualifications

Qualifications Certifications

Educational Background

Bachelors degree in Computer Science, Information Technology, Engineering, or a closely related technical discipline. A Masters degree is preferred.

Mandatory

• AWS Certified Solutions Architect Professional, OR
• Microsoft Certified: Azure Solutions Architect Expert (AZ-305), OR
• Google Cloud Professional Cloud Architect

Preferred

• AWS Machine Learning Specialty, OR Azure AI Engineer Associate, OR Google Professional Machine Learning Engineer
• Azure OpenAI Service / AWS Bedrock / GCP Vertex AI specialisation

Key Competencies Skills

Competencies

FUNCTIONAL COMPETENCIES

• Technical stakeholder advisory - advise client architects and technology leads on AI infrastructure strategy and architectural trade-offs.
• Architecture strategy and roadmap development - assess current-state environments, identify gaps, define target-state designs, and build pragmatic modernisation roadmaps.
• Structured problem-solving under ambiguity - frame AI infrastructure challenges, evaluate trade-offs, and produce actionable recommendations.
• Practice building - codify expertise into reusable frameworks, accelerators, and playbooks.

TECHNICAL COMPETENCIES

• Hyperscaler AI infrastructure - deep expertise in at least one of: AWS (Bedrock, SageMaker, EKS, EC2 GPU), Azure (AI Foundry, Azure OpenAI, Azure ML, AKS), or GCP (Vertex AI, Gemini, GKE), Working knowledge of a second hyperscaler is a strong advantage.
• AI platform architecture - Azure AI Foundry, AWS Bedrock, GCP Vertex AI; model catalog governance, lifecycle management, and self-service enablement patterns.
• Container orchestration - Kubernetes (AKS, EKS, GKE), Docker, and cloud-native infrastructure design for AI workloads.
• Infrastructure as code - Terraform, Ansible, and native hyperscaler IaC tooling; able to set IaC governance standards.
• Inference layer architecture - model serving, routing, caching, latency budgets, and cost optimisation for real-time and batch inference.
• Retrieval and RAG architecture - vector databases, embedding infrastructure, hybrid search, GraphRAG, and multi-tenant isolation at enterprise scale.
• Agentic architecture - tool calling infrastructure, MCP server hosting, agent orchestration frameworks, multi-agent systems, and agent memory patterns.
• MLOps/LLMOps architecture - CI/CD for models, pipeline automation, model versioning, model governance, and evaluation pipelines.
• AI governance and Responsible AI - model governance frameworks, AI guardrails, Managed Identity, RBAC, private connectivity, and Responsible AI controls as applied to AI workloads.
• AI observability - token usage monitoring, per-inference latency, model drift signals, GPU utilisation tracking, and cost attribution.
• Cloud cost optimisation - Well-Architected cost principles applied to AI workloads; compute rightsizing, reserved capacity, GPU utilisation efficiency, and consumption governance.
• AI Compute Infrastructure - familiarity with NVIDIA GPU ecosystem, NVIDIA NIM, DGX platforms, GPU sizing strategies, AI accelerator technologies, and enterprise AI deployment patterns.

Additional Information

Equal Opportunities

Accenture is an equal opportunities employer and welcomes applications from all sections of society and does not discriminate on grounds of race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, or any other basis as protected by applicable law.

Qualification

Experience

1014 years of combined experience across enterprise architecture, cloud infrastructure, and/or AI infrastructure delivery.

• 3+ years designing end-to-end AI infrastructure architectures with architecture ownership.
• Strong expertise in at least ONE hyperscaler (AWS, Azure, or GCP) is mandatory; second hyperscaler is a strong advantage.
• Hands-on design experience with Kubernetes (AKS, EKS, or GKE) and container orchestration at enterprise scale.
• Proficiency in IaC tools (Terraform, Ansible) and native hyperscaler CLIs.
• Experience designing inference serving patterns - provisioned vs pay-as-you-go, model routing, and self-hosted inference on GPU compute.
• Production-scale architectural experience with at least one vector database - sizing, indexing strategy, and multi-tenant isolation.
• Experience designing tool calling and MCP architecture for agentic systems, including security and governance of tool access.
• Experience with multi-agent orchestration frameworks (Semantic Kernel, LangGraph, CrewAI) from an infrastructure-hosting perspective.
• Understanding of MLOps/LLMOps practices - CI/CD for ML, model versioning, model governance, pipeline automation.
• Knowledge of AI observability stacks (Prometheus, Grafana, ELK, OpenTelemetry) and reliability engineering practices.
• Understanding of Well-Architected cost optimisation principles - compute rightsizing, reserved capacity planning, GPU utilisation efficiency, and consumption governance.
• Understanding of Responsible AI principles and AI guardrails at the infrastructure level.
• Enterprise Architecture knowledge (TOGAF or equivalent).
• Architecture design skills at L1L3 levels; proficiency in draw.io, Visio, or Lucid Chart.