AI Architect Quality Intelligence

Zensar Technologies; Pune District; On-site; INR 300,000 - 600,000; Full time

Job summary

Zensar Technologies is seeking a senior AI QA evaluation professional in Pune to lead evaluation-driven development, governing release quality and safety attestations while guiding client engagements and pre-sales efforts.

The role involves managing evaluation pipelines, translating results into deployment decisions, and mentoring future AI specialists within a diversified BFSI, TMT, and manufacturing ecosystem.

Qualifications

• Experience leading evaluation-driven development in AI/ML projects.
• Ability to translate client requirements into solution architectures.
• Experience with presales and client engagements in AI/QA domains.

Responsibilities

• Lead evaluation-driven development and build eval suites.
• Own trajectory grading and safety attestations.
• Translate evaluation results into release decisions.
• Lead client workshops and executive walkthroughs.
• Build reusable accelerators and contribute IP.
• Mentor AI-specialist roles.

Skills

Evaluation governance

Client engagement

Presales leadership

Technical roadmap ownership

Mentoring

Job description

• Design and refine the archetype-specific assurance lifecycles
• Keep the platforms LLM-agnostic and deployable on any client stack — on-prem, cloud or hybrid — as model-mixing and constant provider churn become the market norm.
• Own technical roadmap decisions for accelerators such as the Agentic Foundry (a 25-blueprint reference matrix) and the curated, swap-ready tooling ecosystem around each archetype.

Lead Evaluation-Driven Development

• Build and govern eval suites — ground-truth Q&A sets, LLM-as-judge rubrics, frozen baselines — that gate every release rather than validate it after the fact.
• Own trajectory grading, red/purple/blue-team probes and safety attestations for agentic and generative systems, and drift monitoring and fairness audits for classical ML.
• Translate evaluation results into release decisions: eval-threshold gates, red-team severity floors, canary and shadow deployments, and rollback rehearsals.
• Bring evaluation-driven development practice into client engagements — showing, not just telling, how a live harness beats a one-time audit.

Own Presales and Client Proposals

• Respond to RFPs, RFIs and client proposals across both engines, translating client requirements into a defensible solution architecture and commercial structure.
• Architect engagements across the full ladder — AI QA Assessment, AI QA Transformation, Managed AI QA — and fast-starts such as the LLM Health Check, Agent Stress Test and Compliance Sprint.
• Build estimates, staffing plans and technical win themes that hold up under client and internal scrutiny, across client-managed, risk-reward and Zensar-managed commercial models.

Present and Articulate Value to Clients

• Lead client workshops and technical walkthroughs, including guiding a CIO through the Agentic Foundry in a single session.
• Build and deliver executive trust scorecards and portfolio risk heat maps that make the assurance story board-ready.

Build Practice IP and Mentor the Next Generation

• Contribute reusable accelerators, reference architectures and industry packs across BFSI, TMT, and Manufacturing & Retail back into the practice's IP base.
• Mentor and help build out the practice's emerging AI-specialist roles — Prompt Engineer, LLM-Eval Engineer, Agent Architect, Trajectory Eval Engineer, Knowledge/RAG Engineer, AI Security Analyst and Adversarial Red/Blue Team Lead.