AI Research & Evaluation Lead — Hybrid

City and County of San Francisco; San Francisco (CA); Hybrid; USD 134,000 - 169,000; Full time

Job summary

City and County of San Francisco seeks a Research and Evaluations Lead to advance staff enablement and program evaluation within the Department of Technology. You will guide research on AI and emerging tech, translate insights into actionable recommendations, and work with city departments to design rigorous pilot evaluations.

The role emphasizes qualitative and quantitative methods, collaboration with cross-functional teams, and a focus on responsible adoption of technology in public service.

Qualifications

• Three years in information systems, including system analysis, business process design, development and implementation of business applications or IT project management.
• Associate degree in information systems or related field, or equivalent; substitution options listed in qualifications.
• Experience with qualitative and quantitative research methods; ability to translate findings into product priorities and evaluation plans.
• Experience designing and running program or pilot evaluations with rigorous methods.

Responsibilities

• Conduct qualitative and quantitative research on how City staff use and adopt AI tools to identify friction points and needs.
• Translate research findings into prioritized input for the Emerging Technologies team for tool selection and pilots.
• Assess staff training needs and recommend upskilling approaches for responsible tool use.
• Present findings and recommendations to the Emerging Technologies team, department partners, and City leadership.
• Design and run evaluations of AI and other emerging tech tools to determine effectiveness, safety, and scalability.
• Collaborate with academic and non-profit partners to design rigorous evaluations.
• Combine usage data with surveys and operational metrics to measure outcomes and inform future procurement and rollout decisions.

Skills

Qualitative methods

Quantitative methods

Mixed-methods evaluation

User research

Communication skills

Public service experience

Education

Associate degree in information systems or related field

Substitution: relevant experience or completion of 1010 Information Systems Trainee Program

Tools

Fillout

Qualtrics

Excel/Sheets

SQL / R / Python

Job description

City and County of San Francisco seeks a Research and Evaluations Lead to advance staff enablement and program evaluation within the Department of Technology. You will guide research on AI and emerging tech, translate insights into actionable recommendations, and work with city departments to design rigorous pilot evaluations.

The role emphasizes qualitative and quantitative methods, collaboration with cross-functional teams, and a focus on responsible adoption of technology in public service.