Quality Assurance (QA) - Lead - MiDAS

Robert Bosch Group; Bengaluru; On-site; INR 350,000 - 550,000; Full time

Job summary

Bosch Global Software Technologies Private Limited in Bengaluru seeks an experienced QA lead to shape the AI quality strategy for GenAI solutions. You will own evaluation frameworks focusing on faithfulness, relevancy, and hallucination detection, and drive hybrid test automation across UI/API and LLM outputs.

You will integrate checks into GitHub workflows, lead performance testing with JMeter, and collaborate across DevOps to advance probabilistic quality monitoring using tools like Langfuse

Qualifications

• 8+ years in Software QA
• Automation & tooling expertise
• Expert Python for test tooling
• Hands-on Pytest, Playwright, and JMeter
• GitHub Actions/Workflows for CI/CD
• Core AI/LLM knowledge
• Exposure to GenAI evaluation concepts (faithfulness, relevancy, groundedness)

Responsibilities

• Define and own the evaluation framework for GenAI quality (faithfulness, relevancy, hallucination)
• Architect dual-layer automation: deterministic UI/API tests and probabilistic LLM evaluation
• Integrate automated checks into GitHub Workflows for CI/CD
• Lead JMeter-based performance testing

Skills

QA experience

Automation & tooling

Python

Problem solving

AI/LLM knowledge

GenAI evaluation concepts

LLM logic exposure

RAG systems understanding

Data-driven quality

Education

Bachelors/Masters in CS/SE

Tools

Pytest

Playwright

JMeter

GitHub Actions

Job description

• Legal Entity: Bosch Global Software Technologies Private Limited

Company Description

Bosch Global Software Technologies Private Limitedis a 100% owned subsidiary of Robert Bosch GmbH, one of the world’s leading global supplier of technology and services, offering end‑to‑end Engineering, IT and Business Solutions. With over 27,000+ associates, it’s the largest software development center of Bosch, outside Germany, indicating that it is the Technology Powerhouse of Bosch in India with a global footprint and presence in the US, Europe and the Asia Pacific region.

Job Description

• AI Quality Strategy: Develop and own the evaluation framework for GenAI solutions, focusing on Faithfulness, Relevancy, and Hallucination detection using LLM‑as‑a‑judge frameworks.
• Hybrid Test Automation: Architect a dual‑layered automation suite:
  • Deterministic: E2E UI (Playwright) and API testing (Pytest/Requests).
  • Probabilistic: Automated evaluation of non‑deterministic LLM outputs.
• Shift‑Left Integration: Embed automated quality checks directly into GitHub Workflows, enabling seamless CI/CD.
• Performance & Resilience: Lead JMeter‑based performance testing.

Qualifications

• Experience: 8+ years in Software QA
• Problem Solving: Ability to define "quality" in an ambiguous, non‑deterministic AI landscape.
• Education: Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related field.
• Experience : 8+ years in Software QA
• Mandatory/requires Skills : Automation & Tooling
• Python Mastery: Expert‑level Python skills for building custom test tooling and automation scripts.
• Testing Stack: Hands‑on proficiency with Pytest (API), Playwright (E2E), and JMeter (Performance).
• DevOps: Advanced experience designing and maintaining GitHub Actions/Workflows for automated test execution.
• Core AI & LLM Expertise
• Learning Agility in GenAI: High capability and interest in rapidly mastering AI evaluation concepts. You should be prepared to quickly upskill in automated metrics for LLMs (such as Faithfulness, Relevancy, and Groundedness).
• Exposure to LLM Logic: Basic familiarity with how LLMs function (e.g., prompting, context windows). You should be comfortable exploring and implementing "LLM-as-a-Judge" strategies, where high‑reasoning models help grade application‑specific outputs.
• Orientation toward RAG Systems: Interest in understanding the mechanics of Retrieval‑Augmented Generation (RAG). You will be responsible for defining how we validate the accuracy of data retrieved from our engineering context catalogues and vector databases.
• Data‑Driven Quality Mindset: A strong desire to move beyond binary "Pass/Fail" results toward probabilistic quality monitoring, utilizing tools like Langfuse to analyze live traces and performance trends.
• Preferred Skills :

Additional Information

Why Join MiDAS?

You won’t just be testing software; you will be defining the quality standards for the future of AI-First Engineering. Your work will directly impact the speed and reliability of vehicle software development globally.