Staff Quality Engineer — Qira AI Platform

Lenovo; Bengaluru; On-site; INR 4,500,000 - 7,200,000; Full time

Job summary

Lenovo in Bengaluru, India is seeking a Staff AI Quality Engineer for the Qira AI Platform to lead test strategy and architectural validation across cross-device ecosystems. You will design automated quality platforms, enforce non-functional standards for privacy, security, accessibility, internationalization, and device thermals, and partner with PMO and Product Management on release gates.

The role requires 8+ years in software quality, hands-on automation, and experience with LLMs, agentic AI

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field.
• 8+ years of progressive software quality engineering and automation across multi-device ecosystems.
• 3+ years leading quality strategy and automated evaluation frameworks for AI/LLM workflows.

Responsibilities

• Lead test strategy and architectural validation for cross-device AI platform.
• Design automated evaluation methodologies for multi-step agent reasoning and rollback mechanisms.
• Develop scalable test harnesses and CI/CD gating for non-deterministic AI models.
• Profile on-device performance, memory, battery and thermals across hardware tiers.
• Coordinate with PMO and Product Management on release acceptance criteria.

Skills

Python

C++

C#

Java

Test automation

Multi-device testing

LLM testing

Education

Bachelor’s or Master’s degree in Computer Science/Software Engineering

Tools

LangSmith

TruLens

RAGAS

DeepEval

Job description

We are Lenovo. We do what we say. We own what we do. We WOW our customers.

Lenovo is a US$83 billion revenue global technology powerhouse, ranked #153 in the Fortune Global 500, and serving millions of customers every day in 180 markets. Focused on a bold vision to deliver Smarter Technology for All, Lenovo has built on its success as the world’s largest PC company with a full-stack portfolio of AI-enabled, AI-ready, and AI-optimized devices (PCs, workstations, smartphones, tablets), infrastructure (server, storage, edge, high performance computing and software defined infrastructure), software, solutions, and services. Lenovo’s continued investment in world-changing innovation is building a more equitable, trustworthy, and smarter future for everyone, everywhere. Lenovo is listed on the Hong Kong stock exchange under Lenovo Group Limited (HKSE: 992) (ADR: LNVGY).

This transformation together with Lenovo’s world-changing innovation is building a more inclusive, trustworthy, and smarter future for everyone, everywhere. To find out more visit www.lenovo.com , and read about the latest news via our StoryHub .

Description and Requirements

About the Role

As a Staff AI Quality Engineer for the Qira AI Platform, you will lead the technical test strategy and architectural validation for Lenovo’s cross-device agentic AI ecosystem. You will serve as the technical authority on evaluating non-deterministic systems, designing automated quality platforms that operate across personal computing form factors (PCs, tablets, and smartphones). You will drive non-functional quality standards—ensuring that privacy, security, accessibility, internationalization, and device thermals/battery efficiency are safeguarded—while acting as a crucial release partner alongside Product Management and PMO leads.

What Will You Do

• Test Automation Architecture: Architect scalable, distributed, cross-platform test automation infrastructure capable of executing synchronized, multi-device end-to-end tests across Windows and Android devices without reliance on UI flakiness.
• Agentic AI Quality Frameworks: Design automated evaluation methodologies to validate multi-step agent reasoning, tool-calling graphs, autonomous error recovery, and transactional rollback mechanisms; establish statistical release gates for non-deterministic model behaviors.
• Hardware & Runtime Resource Profiling: Formulate automated performance and endurance soak suites to benchmark on-device SLM execution against cloud fallback paths, monitoring thermal throttling, memory leaks, and battery draw across hardware device tiers.
• Non-Functional Quality Standards: Lead automated verification across non-functional pillars: red-teaming indirect prompt injection vulnerabilities, securing inter-process data barriers, standardizing automated accessibility (screen-reader verbosity, dynamic token streaming), and validating semantic fidelity across international locales.
• CI/CD Quality Pipelines & Gating: Drive the technical implementation of shift-left gating in continuous integration pipelines, establishing automated failure analytics, regression detection, and flakiness isolation frameworks.
• Cross-Functional Peer Leadership: Serve as a core technical gatekeeper alongside Product Management and PMO; establish release acceptance criteria, lead data-informed risk assessments during schedule compressions, and influence platform design for enhanced testability.

Basic Qualifications

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related technical field.
• 8+ years of progressive software quality engineering, test architecture, and automation experience covering multi-device hardware ecosystems (Windows and Android).
• 3+ years of experience leading quality strategy and automated evaluation frameworks for LLMs, agentic AI workflows, or conversational platforms.
• Advanced programming and architecture skills in Python, C++, C#, or Java, with a track record of building scalable test harnesses and mock environments for OS IPC, APIs, and background daemons.
• Proven experience architecting automated non-functional test frameworks: deep profiling of device performance (NPU/GPU/CPU, battery mA/h, memory), security red-teaming/prompt-injection defense, automated accessibility (WCAG), and i18n semantic evaluation.
• Deep technical expertise in CI/CD pipeline architecture, test orchestration, and statistical quality gating.

Preferred Qualifications

• Hands-on expertise implementing and tuning non-deterministic evaluation platforms (e.g., RAGAS, DeepEval, LangSmith, TruLens) in production pipelines.
• Direct experience testing local NPU runtimes and quantized models (INT4/INT8 vs. FP16) on client silicon (Qualcomm, Intel, AMD).
• Background designing automated testing for multimodal inputs (synchronized voice injection, screen-capture parsing, synthetic system events).
• Demonstrated ability to resolve complex release-gating disputes with Product Management and PMO using objective risk metrics and clear mitigation trade-offs.

Additional Locations

• India - Karnātaka - Bangalore
• India - Karnātaka - BANGALORE

If you require an accommodation to complete this application, please contact [e-mail]