QA Engineer: AI & LLM Test Automation

Intercontinental Exchange (ICE); Atlanta (GA); On-site; USD 95,000 - 130,000; Full time

Job summary

Intercontinental Exchange, Inc. Mortgage Technology is seeking a Quality Engineering professional to advance our automated testing for LLM-based applications within the Collections and Loss Mitigation platform in Atlanta.

The role requires a Bachelor’s degree in STEM, 3+ years in QA/QA automation, and strong Python skills, with experience using Agentic AI tools and developing AI prompts to validate AI-generated outputs.

Qualifications

• Bachelor’s degree in STEM fields.
• 3+ years in QA and QA automation.
• Experience using Agentic AI tools to build test automation tools.
• Proficiency in Python.
• Ability to learn quickly and adapt to new technologies and processes, with a focus on LLM related technologies.
• Excellent communication and teamwork skills.

Responsibilities

• Implement automated test frameworks and take ownership of existing frameworks for LLM based applications.
• Implement automated tests that use AI for reasoning on the correctness of LLM generated information.
• Improve testing methodologies for LLM based applications.
• Participate in designing and improving prompts used in LLM based applications.

Skills

QA automation

Python

LLM testing

Agentic AI tools

Testing frameworks

Communication skills

Education

Bachelor’s degree in STEM fields

Tools

Agentic AI tools

Job description

Intercontinental Exchange, Inc. Mortgage Technology is seeking a Quality Engineering professional to advance our automated testing for LLM-based applications within the Collections and Loss Mitigation platform in Atlanta.

The role requires a Bachelor’s degree in STEM, 3+ years in QA/QA automation, and strong Python skills, with experience using Agentic AI tools and developing AI prompts to validate AI-generated outputs.