QA Automation Engineer — AI/LLM Testing & Frameworks

Intercontinental Exchange Holdings, Inc.; Jacksonville (FL); On-site; USD 90,000 - 130,000; Full time

Job summary

Intercontinental Exchange, Inc. is seeking a QA Engineer to help build and own automated tests for LLM-based applications within ICE Mortgage Technology. You will implement robust testing frameworks, write AI-augmented tests, and evolve QA approaches to ensure quality across the platform.

The role emphasizes quick learning, collaboration, and strong problem-solving in a dynamic financial services environment based in Jacksonville, FL. A STEM degree and 3+ years of QA experience are preferred.

Qualifications

• Bachelor’s degree in STEM fields required.
• 3+ years in QA and QA automation.
• Proficiency in Python.
• Experience with Agentic AI tools for test automation.
• Strong communication and teamwork skills.

Responsibilities

• Implement automated test frameworks and own existing frameworks for LLM-based applications.
• Create automated tests that use AI for reasoning about LLM outputs.
• Improve testing methodologies for LLM-based applications.
• Contribute to prompts design and improvement for LLM-based apps.

Skills

Python

QA automation

Agentic AI tools

Communication

Teamwork

Learning agility

Problem solving

Education

Bachelor’s degree in STEM

Tools

Selenium

Python testing frameworks

Job description

Intercontinental Exchange, Inc. is seeking a QA Engineer to help build and own automated tests for LLM-based applications within ICE Mortgage Technology. You will implement robust testing frameworks, write AI-augmented tests, and evolve QA approaches to ensure quality across the platform.

The role emphasizes quick learning, collaboration, and strong problem-solving in a dynamic financial services environment based in Jacksonville, FL. A STEM degree and 3+ years of QA experience are preferred.