QA Engineer

ICE; Atlanta (GA); On-site; USD 80,000 - 130,000; Full time

Job summary

ICE is seeking a QA Engineer to test LLM-based software, build and maintain automated tests, and help optimize prompts. You will join a team focused on delivering reliable AI-driven solutions and ensuring quality across multiple projects.

The role requires 3+ years in QA automation, strong Python skills, and experience with Agentic AI tools, with an emphasis on continuous learning and collaboration.

Qualifications

• 3+ years in QA and QA automation.
• Proficiency in Python programming.
• Experience with Agentic AI tools for test automation.
• Ability to learn quickly and adapt to new tech.
• Strong communication and teamwork in technical contexts.

Responsibilities

• Implement automated test frameworks and maintain existing ones for LLM-based apps.
• Create automated tests using AI to assess correctness of LLM outputs.
• Improve testing methodologies for LLM-driven applications.
• Contribute to designing and refining prompts used in LLM systems.

Skills

Python

QA automation

Agentic AI tools

Communication

Teamwork

Learning agility

Education

Bachelor’s degree in STEM

Job description

Job Purpose

We are seeking a QA Engineer with experience in testing LLM based software. As a member of the QA team, you will have the opportunity to participate in various projects where LLMs are used to solve complex engineering problems. You will participate in creating and executing automated tests for such applications. You will also participate in the creation and improvement of prompts driving these applications.

Overview

Job Purpose

We are seeking a QA Engineer with experience in testing LLM based software. As a member of the QA team, you will have the opportunity to participate in various projects where LLMs are used to solve complex engineering problems. You will participate in creating and executing automated tests for such applications. You will also participate in the creation and improvement of prompts driving these applications.

Responsibilities

• Implement automated test frameworks and take ownership of existing frameworks for LLM based applications.
• Implement automated tests that use AI for reasoning on the correctness of LLM generated information.
• Improve testing methodologies for LLM based applications.
• Participate and designing and improving prompts used in LLM based applications.

Knowledge And Experience

• Bachelor’s degree in STEM fields.
• 3+ years in QA and QA automation.
• Experience using Agentic AI tools to build test automation tools.
• Proficiency in Python.
• Ability to learn quickly and adapt to new technologies and processes, with a focus on keeping up to date with the latest developments in LLM related technologies.
• Ability to independently learn new complex business domains.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork skills.
• Ability to communicate fluently and professionally in complex technical and business contexts.

Preferred Knowledge And Experience

• Knowledge of fixed income markets and financial instruments.

Intercontinental Exchange, Inc. is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to legally protected characteristics.