Principal AI Architect

Trianz; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Trianz seeks a Principal AI Architect to own the complete technical architecture for deploying large language models inside customer environments with full sovereignty and control. The role emphasizes production LLM serving, GPU/CPU optimization, and Kubernetes-scale operations across multi-cloud, on-premises, and air-gapped setups.

You will shape model selection, serving, routing, monitoring and governance across AWS, Azure, GCP, RedHat OpenShift, VMware, and air-gapped infrastructure, acting

Qualifications

• Extensive experience deploying LLMs in real enterprise environments.
• Hands-on with production LLM serving frameworks and GPUs.
• Ability to design governance, security and data residency for regulated environments.
• Proven track record in architecture decisions across multi-cloud and on-premise.

Responsibilities

• Define private LLM serving stack and production topology.
• Architect cross-cloud model serving with provider-agnostic design.
• Lead governance, monitoring, and guardrail architecture for AI systems.
• Specify per-tenant isolation and security boundaries for deployments.
• Drive DevSecOps pipelines and self-service deployment standards.
• Guide optimization of CPU/GPU routing and quantization strategies.

Skills

LLM Serving

Kubernetes

Multi-Cloud

Open-Source ML

Security & Compliance

GPU Optimization

Inference Frameworks

Tools

vLLM

TensorRT-LLM

Triton

RedHat OpenShift

Kubernetes

VMware

Job description

Trianz is an applied AI solutions company that accelerates customer business transformation through AI powered "Transformation Services as a Software Model". With 25+ years of transforming enterprises, we've evolved to a product-led, platform-driven organization serving global enterprises across Financial Services, Insurance, Healthcare, Hi-Tech, Manufacturing, and other industries.

With global presence across 4 continents, our platform portfolio under the unified Concierto brand delivers end-to-end transformations including solutions for Migrate, Manage, Maximize, Modernize, Insights & Agentic AI, and SecOps - delivered through strategic partnerships with leading hyperscalers.

We're building the premier innovation-led organization in the digital transformation space through AI-first methodologies and data-driven excellence - RevolutionAIzing Transformations.

Role Overview

Trianz is scaling its private and sovereign AI capabilities to meet enterprise demand for on-premises, air-gapped, and provider-agnostic LLM deployment. We're seeking a Principal AI Architect to own the complete technical architecture for deploying large language models inside customer environments — not consuming cloud APIs but designing the systems that run LLMs with full sovereignty and control.

This is a pure individual contributor role with architectural authority. You will define how models are selected, served, routed, monitored, and governed across multi-cloud, on-premises, and hybrid environments. You'll work with enterprise customers running on AWS, Azure, GCP, RedHat OpenShift, VMware, and air-gapped infrastructure.

This role demands deep hands-on expertise in production LLM serving, GPU/CPU optimization, Kubernetes at scale, and the ability to make architecture decisions in complex, regulated environments. You'll be the authority that production AI engineers look to for guidance.

Key Responsibilities

LLM Serving Architecture

• Design the complete private LLM serving stack: model selection, serving frameworks (vLLM, TensorRT-LLM, Triton), and production topology


• Define intelligent CPU vs GPU routing policies based on latency, cost, throughput, and model characteristics


• Architect model serving for diverse hardware: NVIDIA GPUs, Intel Xeon CPUs, AMD EPYC, and heterogeneous clusters


• Evaluate open-source models (Llama, Mistral, Qwen, Phi) against closed models for enterprise use cases


• Architect multi-cloud model serving (AWS, Azure, GCP) with provider-agnostic, no-lock-in design


• Design on-premises serving for enterprise customers: RedHat OpenShift, VMware, containerized model serving on customer hardware


• Architect air-gapped, sovereign deployment patterns with data-residency compliance


• Design per-tenant model isolation and security boundaries

Model Lifecycle & Operations

• Define release architecture for model versions: staged rollout, deployment gates, rollback, and promotion strategies


• Design the LLM governance framework: model behavior monitoring, inference audit logging, guardrail architecture


• Architect DevSecOps pipeline and self-service deployment automation standards


• Design model quantization strategies (GPTQ, AWQ, GGUF) and routing trade-offs for cost and performance

Architecture Authority & Governance

• Produce architecture sign-off documents and review all AI system designs before implementation


• Define standards for production LLM deployment, security, compliance, and scalability


• Establish best practices for GPU FinOps and cost modeling across mixed inference fleets


• Lead technical decisions on inference schedulers and custom optimization approaches

Ideal Candidate Profile

Experience: 15 + years

Production LLM Deployment:

• Deployed LLMs to production in a real enterprise environment — not just API consumption


• Hands-on with vLLM, TensorRT-LLM, or Triton in a production serving context


• Designed CPU cluster inference (Intel Xeon / AMD EPYC) for open-source models


• 8+ years in AI/software architecture with at least 3 years in production LLM systems

Infrastructure & Scale:

• Kubernetes at production scale (EKS, AKS, GKE, or OpenShift) — not just local k8s


• Designed multi-cloud, provider-agnostic AI architectures


• Experience with model quantization (GPTQ, AWQ, GGUF) and routing trade-offs

Technical Depth

• Deep understanding of LLM serving frameworks and inference optimization


• Experience with GPU memory management, batching, and throughput optimization


• Understanding of model serving topologies: single-model, ensemble, and traffic-driven routing


• Proficiency in containerization and orchestration at enterprise scale


• Knowledge of security, compliance, and data residency in regulated environments

Mindset & Fit

• Architect-level thinking: ability to make decisions with incomplete information and trade-offs


• Comfort with ambiguity in emerging sovereign AI landscape


• Driven by technical depth and system design excellence, not titles


• Collaborative: works across teams to translate business requirements into architecture


• Bias toward open-source, extensible solutions over proprietary managed services

Nice to Have

• RedHat OpenShift for on-premises model serving


• NVIDIA Dynamo, SGLang, or custom inference schedulers


• GPU FinOps and cost modeling for mixed CPU/GPU inference fleets


• Sovereign AI or regulated-industry deployment experience (healthcare, finance, defense)


• Intel OpenVINO or AMD ROCm for CPU-optimized inference


• Experience with model monitoring, observability, and telemetry at scale


• Previous role at hyperscalers (AWS, Azure, GCP) in the AI/ML infrastructure space


• Background in edge computing, IoT, or on-premises AI systems

Why Join Trianz?

• Architectural Impact: Own the complete private AI infrastructure for Fortune 500 enterprises. Your decisions influence how LLMs run in the most security-conscious, regulated environments globally.


• Technical Excellence: Work with cutting-edge inference frameworks, open-source models, and heterogeneous hardware. Design systems that work across AWS, Azure, GCP, on-prem, and air-gapped environments — not just API consumption.


• Sovereign AI Leadership: Lead the charge in sovereign AI deployment. Help enterprises reclaim control of their AI infrastructure while maintaining the flexibility to scale globally.


• Zero Bureaucracy: Pure IC role with architectural authority. No slow approval cycles — your design decisions move fast into production.


• Enterprise Scale: Work on transformations across Fortune 500 organizations. See your architecture deployed across continents, industries, and mission-critical use cases.


• Growth Through Ambiguity: Thrive in the emerging sovereign AI space where problems are complex, standards are evolving, and your expertise will define the industry.