QA Automation Engineer – LLM/AI Testing

ICE Clear Europe Limited; Georgia; On-site; USD 85,000 - 125,000; Full time

Job summary

ICE Mortgage Technology is building a world-class Quality Engineering organization to advance Collections and Loss Mitigation while supporting homeowners’ financial journeys. The role focuses on implementing automated test frameworks for LLM-based applications and elevating quality across the platform.

We seek a Bachelor’s degree in STEM with 3+ years in QA and QA automation, proficiency in Python, and experience with Agentic AI test tools.

Qualifications

• Bachelor's degree in STEM fields is required.
• 3+ years in QA and QA automation.
• Experience using Agentic AI tools to build test automation tools.
• Proficiency in Python and automated testing.
• Strong problem solving and attention to detail.
• Excellent communication and teamwork abilities.

Responsibilities

• Implement automated test frameworks and own existing frameworks for LLM-based applications.
• Implement automated tests that use AI for reasoning on LLM outputs.
• Improve testing methodologies for LLM-based applications.
• Participate in designing and improving prompts used in LLM applications.

Skills

Python

QA automation

AI test tools

Communication skills

Teamwork

Problem solving

Fast learner

Adaptability

Education

Bachelor's degree in STEM fields

Tools

Agentic AI tools

Job description

ICE Mortgage Technology is building a world-class Quality Engineering organization to advance Collections and Loss Mitigation while supporting homeowners’ financial journeys. The role focuses on implementing automated test frameworks for LLM-based applications and elevating quality across the platform.

We seek a Bachelor’s degree in STEM with 3+ years in QA and QA automation, proficiency in Python, and experience with Agentic AI test tools.