Analyst, QA

Intercontinental Exchange Holdings, Inc.; Jacksonville (FL); On-site; USD 90,000 - 130,000; Full time

Job summary

Intercontinental Exchange, Inc. is seeking a QA Engineer to help build and own automated tests for LLM-based applications within ICE Mortgage Technology. You will implement robust testing frameworks, write AI-augmented tests, and evolve QA approaches to ensure quality across the platform.

The role emphasizes quick learning, collaboration, and strong problem-solving in a dynamic financial services environment based in Jacksonville, FL. A STEM degree and 3+ years of QA experience are preferred.

Qualifications

• Bachelor’s degree in STEM fields required.
• 3+ years in QA and QA automation.
• Proficiency in Python.
• Experience with Agentic AI tools for test automation.
• Strong communication and teamwork skills.

Responsibilities

• Implement automated test frameworks and own existing frameworks for LLM-based applications.
• Create automated tests that use AI for reasoning about LLM outputs.
• Improve testing methodologies for LLM-based applications.
• Contribute to prompts design and improvement for LLM-based apps.

Skills

Python

QA automation

Agentic AI tools

Communication

Teamwork

Learning agility

Problem solving

Education

Bachelor’s degree in STEM

Tools

Selenium

Python testing frameworks

Job description

Overview

Job Purpose

At ICE Mortgage Technology, we are building a world-class Quality Engineering organization that will help elevate our Collections and Loss Mitigation offerings while strengthening our ability to support homeowners throughout their financial journey. At the same time, we are empowering loan servicers with more effective tools, automation, and insights to identify, prioritize, and focus on homeowners who require the greatest level of support. By enhancing quality across our platform, we can help ensure that engagement and assistance efforts are directed where they are needed most, improving outcomes for homeowners, servicers, and investors alike.

ICE has established a strong Collections and Loss Mitigation offering within the industry. We are now focused on advancing these capabilities through continued innovation, operational excellence, and an unwavering commitment to quality, while further integrating them across the broader loan lifecycle and default servicing process. Our vision is to provide a seamless, end-to-end experience that enables servicers to engage homeowners proactively, make informed decisions, and deliver assistance more effectively. Building a best-in-class Quality Engineering organization is a foundational step in achieving that vision, and these positions will play a critical role in shaping the next generation of our platform and customer experience.

Responsibilities

• Implement automated test frameworks and take ownership of existing frameworks for LLM based applications.
• Implement automated tests that use AI for reasoning on the correctness of LLM generated information.
• Improve testing methodologies for LLM based applications.
• Participate and designing and improving prompts used in LLM based applications.

Knowledge and Experience

• Bachelor’s degree in STEM fields.
• 3+ years in QA and QA automation.
• Experience using Agentic AI tools to build test automation tools.
• Proficiency in Python.
• Ability to learn quickly and adapt to new technologies and processes, with a focus on keeping up to date with the latest developments in LLM related technologies.
• Ability to independently learn new complex business domains.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork skills.
• Ability to communicate fluently and professionally in complex technical and business contexts.

Preferred Knowledge and Experience

• Knowledge of fixed income markets and financial instruments.

Intercontinental Exchange, Inc. is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to legally protected characteristics.