AI Software Test Engineer - Mid Level (Onsite/Hybrid)

Spectraforce Technologies, Inc.; Ann Arbor (MI); On-site; USD 90,000 - 120,000; Full time

Job summary

Spectraforce Technologies, Inc. is seeking a Mid-Level AI Software Test Engineer in Ann Arbor, MI for 18 months, onsite for six months and then a 4:1 onsite-to-remote schedule.

You will design, automate, and execute testing strategies for AI-powered applications, ML systems, agents, copilots, and generative AI solutions. You should have a strong software testing background, experience building automated test frameworks, and interest in AI technologies such as LLMs, RAG systems, MCP integrations,

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Information Systems, or related field.
• 3-6 years of software testing, QA automation, or quality engineering experience.
• Experience developing automated test solutions using Python, Java, JavaScript/TypeScript, or C#.
• Experience with API testing and automation tools.
• Strong understanding of test automation, SDLC, Agile, and CI/CD pipelines.
• Experience testing distributed systems, web applications, and APIs.

Responsibilities

• Develop comprehensive testing strategies for AI applications, platforms, and services.
• Validate AI model outputs for accuracy, consistency, reliability, and safety.
• Design and execute functional, integration, end-to-end, regression, and performance tests for AI solutions.
• Create test cases for prompt-driven, agentic, and retrieval-based AI workflows.
• Validate AI guardrails, business rules, permissions, and governance controls.
• Perform adversarial, negative, and edge-case testing to identify model failures and hallucinations.
• Build and maintain automated test frameworks for AI applications.
• Develop automated evaluation pipelines for AI responses and workflows.
• Integrate AI testing into CI/CD pipelines.
• Implement automated quality scoring and regression detection.
• Create reusable test data, mocks, simulators, and validation frameworks.
• Test AI agents, workflows, APIs, MCP integrations, and tool-calling capabilities.
• Validate integrations with external systems, data sources, and enterprise services.
• Verify performance, reliability, scalability, and resiliency of AI workloads.
• Execute load and stress testing for AI services.
• Partner with software engineers, AI engineers, product owners, architects, and security teams.
• Participate in design reviews and provide quality feedback during development.
• Contribute to test strategy, quality standards, and best practices.
• Support production readiness reviews and defect triage activities.

Skills

Python

Java

JavaScript/TypeScript

C#

API Testing

Test Automation

SDLC

Agile

CI/CD

Distributed Systems

Web Applications

APIs

Selenium

Playwright

Education

Bachelor's degree in Computer Science or related field

Tools

Selenium

Playwright

Postman

JUnit

PyTest

GitHub Actions

Jenkins

Docker

Kubernetes

Job description

Spectraforce Technologies, Inc. is seeking a Mid-Level AI Software Test Engineer in Ann Arbor, MI for 18 months, onsite for six months and then a 4:1 onsite-to-remote schedule.

You will design, automate, and execute testing strategies for AI-powered applications, ML systems, agents, copilots, and generative AI solutions. You should have a strong software testing background, experience building automated test frameworks, and interest in AI technologies such as LLMs, RAG systems, MCP integrations,