Senior AI QA Engineer | Test Automation & Benchmarks

Solid Company; Kraków; Hybrid; PLN 180,000 - 240,000; Full time

Benefits offered by this job

Hybrid work model

Office collaboration spaces and onsite

Life insurance

Development budget with language and培训

MyBenefit platform with Multisport and

Job summary

Solid is seeking an experienced software engineer to build robust test automation frameworks for MCP-native applications and to develop benchmarks for AI-enabled features. You will work across the stack and collaborate with developers to improve quality in both development and production environments.

Ideal candidates bring 6+ years of software engineering or QA experience, strong Java/Python/React/TypeScript skills, and a track record in AI-driven testing and LLM benchmarking within GCP

Qualifications

• 6+ years of professional software engineering and/or test automation experience.
• Solid experience with Java, Python, React, TypeScript and CSS.
• Experience in AI-driven testing, automated test generation and modern QA methodologies.
• Experience building benchmarks for LLM-based systems.
• Knowledge of the AI software development life cycle, Vertex AI and Gemini models.
• Experience operating within GCP.
• Strong English communication, analytical thinking and problem-solving abilities.

Responsibilities

• Build robust test automation frameworks for MCP-native applications in ChatGPT.com and Claude.ai.
• Prepare benchmarks for AI functions integrated into applications and maintain the test inventory.
• Assess the application portfolio from a security perspective.
• Use Gemini models to develop testing strategies and generate test cases for MCP endpoints and chat UI widgets.
• Diagnose and resolve quality and integration issues in development and production.
• Collaborate with developers, document testing methodologies and demonstrate workflows to the Product Owner during weekly sessions.

Skills

Java

Python

React

TypeScript

CSS

English communication

Education

B.S. or M.S. in Computer Science/IT/Engineering

Tools

Vertex AI

Gemini models

Google Cloud (GCP)

Job description

Solid is seeking an experienced software engineer to build robust test automation frameworks for MCP-native applications and to develop benchmarks for AI-enabled features. You will work across the stack and collaborate with developers to improve quality in both development and production environments.

Ideal candidates bring 6+ years of software engineering or QA experience, strong Java/Python/React/TypeScript skills, and a track record in AI-driven testing and LLM benchmarking within GCP