AI Engineer: Prompting, QA & Observability

Quadrant IQ Solutions LLC

Town of Texas (WI)

On-site

USD 120,000 - 190,000

Full time

4 days ago

Be an early applicant

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Job summary

Quadrant IQ Solutions LLC is seeking an AI Engineer to design, test, optimize, and support AI-powered agent experiences across business workflows. This role combines prompt engineering, agent orchestration, QA testing, evaluation framework development, and AI observability.

You will design prompts and systems, conduct experiments, and collaborate with engineering to deploy updates through established SDLC processes. The ideal candidate has strong AI, testing, and collaboration skills.

Qualifications

Bachelor's degree in CS, engineering, information systems, or related field.
5–10 years in software QA, automation testing, or related technical roles.
1+ years hands-on with Generative AI, LLMs, or conversational AI.
Experience with Azure DevOps or similar Agile delivery platforms.
Experience designing and executing structured test plans.
Strong analytical and troubleshooting skills.
Excellent written and verbal communication skills.

Responsibilities

Design, test, optimize, and support AI-powered agent experiences.
Lead prompt engineering and agent design across workflows.
Develop evaluation frameworks and scoring for AI output quality.
Create test cases for accuracy, safety, tone, instruction following, and consistency.
Collaborate with engineering to deploy updates via SDLC and Azure DevOps.
Maintain documentation for prompts, evaluation methodologies, and agent configurations.
Participate in Agile ceremonies and communicate findings to stakeholders.
Monitor AI observability platforms and identify improvement opportunities.