AI/LLM QA Automation Engineer (Burgess Hill)

TEKsystems; Burgess Hill; On-site; GBP 45,000 - 60,000; Part time

Job summary

TEKsystems in Burgess Hill, UK is seeking an Automation Tester to design and implement testing strategies for AI/LLM-based systems. You will develop evaluation frameworks for AI outputs and assess accuracy, relevance, and edge cases, while ensuring consistency and reliability.

Collaborating closely with engineering, product, and data teams, you will help define quality standards for non-deterministic AI systems and help drive automated evaluation techniques in a contract setting.

Qualifications

• Design and implement testing strategies for AI/LLM-based systems.
• Develop and apply evaluation frameworks for AI outputs.
• Assess and validate: accuracy and relevance of responses; consistency and reliability; hallucinations and edge cases.
• Work with LLM-as-judge patterns and automated evaluation techniques.
• Define quality standards for non-deterministic AI systems.
• Collaborate closely with engineering, product, and data teams.

Responsibilities

• Design and implement testing strategies for AI/LLM-based systems.
• Develop and apply evaluation frameworks for AI outputs.
• Assess and validate: accuracy and relevance of responses; consistency and reliability; hallucinations and edge cases.
• Work with LLM-as-judge patterns and automated evaluation techniques.
• Define quality standards for non-deterministic AI systems.
• Collaborate with engineering, product, and data teams.

Skills

Testing strategies

Evaluation frameworks

LLM-as-judge patterns

Quality standards

Automation

QA collaboration

Tools

Java

Kotlin

Kafka

Job description

TEKsystems in Burgess Hill, UK is seeking an Automation Tester to design and implement testing strategies for AI/LLM-based systems. You will develop evaluation frameworks for AI outputs and assess accuracy, relevance, and edge cases, while ensuring consistency and reliability.

Collaborating closely with engineering, product, and data teams, you will help define quality standards for non-deterministic AI systems and help drive automated evaluation techniques in a contract setting.