Public Sector AI Solutions Analyst - Evaluation

Public Consulting Group; Lansing (MI); On-site; USD 56,000 - 115,000; Full time

Benefits offered by this job

Medical and dental care

401k

PTO

Parental leave

Bereavement leave

Job summary

Public Consulting Group is seeking an AI Solutions Analyst to support design, delivery, and monitoring of AI-enabled solutions in state government environments. You will translate policy needs into structured requirements and assess evaluation data to improve system output quality.

The role blends business analysis with hands-on AI fluency—bridging stakeholders and technical teams, evaluating prompts, and balancing performance, cost, and explainability within enterprise contexts.

Qualifications

• Bachelor’s degree or equivalent in a relevant field.
• 4+ years of experience as a Business Analyst, Solutions Analyst, or Consultant.
• Experience translating complex needs into structured deliverables.
• Client-facing experience in requirements sessions and workshops.
• Working knowledge of basic statistics and distributions.
• Hands-on experience iterating on prompts for LLM-based systems.
• Familiarity with AI systems, behavior and limitations.
• Technical understanding of system architecture, data flows, APIs, and integrations.

Responsibilities

• Support and lead client requirements sessions and discovery discussions.
• Translate policy and operational workflows into structured requirements.
• Serve as a bridge between stakeholders and technical teams.
• Support design of AI-enabled solutions and define use cases.
• Develop and document AI workflows.
• Design evaluation test sets with representative samples across case types.
• Analyze evaluation results to identify underperformance and guide improvements.
• Iterate prompts and instructions to address root causes identified through evaluation analysis.
• Identify tradeoffs related to performance, cost, and explainability.
• Partner with technical leads on tuning cycles within review processes.
• Support broader program operations and cross-workStream coordination.

Skills

Strong communication

Client-facing experience

Prompts for LLMs

Statistical reasoning basics

AI system knowledge

System architecture understanding

Data flows and APIs

Education

Bachelor’s degree from an accredited institution

Tools

AWS Bedrock

Azure

Job description

Public Consulting Group is seeking an AI Solutions Analyst to support design, delivery, and monitoring of AI-enabled solutions in state government environments. You will translate policy needs into structured requirements and assess evaluation data to improve system output quality.

The role blends business analysis with hands-on AI fluency—bridging stakeholders and technical teams, evaluating prompts, and balancing performance, cost, and explainability within enterprise contexts.