Program Lead, AI Evaluation & Embedded Assessments

Transluce; San Francisco (CA); On-site; USD 250,000 - 350,000; Full time

Benefits offered by this job

Travel opportunities

Job summary

Transluce, a fast-moving nonprofit research lab in San Francisco, seeks a Technical Program Manager to scope, organize, and direct the work of our technical staff across external engagements, including on-site evaluations and rapid-response analysis. You’ll own each engagement end to end from scoping to dissemination.

You will work with our governance and technical teams to protect researcher time, turn engagements into repeatable playbooks, and travel as needed.

Qualifications

• Understanding of AI evaluations and ability to engage with technical teams on design and results.
• Track record of managing cross-functional teams, for example in a tech company, consulting firm, or fast-paced startup.
• Experience engaging external partners such as frontier AI companies, government actors, or civil society organizations.
• Ability to work through ambiguity and deliver against complex, evolving goals on short timelines, building workflows where none exist.
• Ability to translate technical research for a range of audiences.
• Exceptional communication and relationship skills, low ego, openness to giving and receiving feedback.

Responsibilities

• Define the scope of each engagement, set milestones, and negotiate terms with support from operations.
• Organize the technical team's work, manage dependencies, and keep everyone aligned through frequent touchpoints.
• Act as a sounding board on strategic and methodological questions, reflecting engagement needs and regulatory considerations.
• Own the day-to-day partner relationship, building trust and protecting Transluce’s interests and the team’s time.
• Engage with evaluation results and technical analysis to understand work, identify issues, and drive engagement.
• Present deliverables to partners and summarize results for broader audiences, including blog posts, papers, and briefings.

Skills

AI evaluations understanding

Cross-functional leadership

External partnerships

Ambiguity management

Communication skills

Technical program management

Python

PyTorch

Hugging Face

Inspect

Tools

Python

pandas

SQL

PyTorch

Hugging Face

Inspect

Job description

Transluce, a fast-moving nonprofit research lab in San Francisco, seeks a Technical Program Manager to scope, organize, and direct the work of our technical staff across external engagements, including on-site evaluations and rapid-response analysis. You’ll own each engagement end to end from scoping to dissemination.

You will work with our governance and technical teams to protect researcher time, turn engagements into repeatable playbooks, and travel as needed.