Senior Product Manager - EMEA - Remote (Fully Remote)

Pencil; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

25 days PTO

Public holidays

Health insurance

Wellness stipend

Parental leave

Fully remote

London office access

Job summary

Pencil is an advertising technology company using Generative AI to redefine content creation for brands. Founded in 2018 by former Google, Facebook and Uber employees, it is backed by Sequoia and now part of The Brandtech Group, fostering a fast-moving, human-centric culture.

You will own the Evals Quality charter, defining success metrics and improving creative quality across Pencil's agentic system. You will partner with commercial teams to provide evidence for client RFPs and enablement

Qualifications

• Define core quality metrics and success criteria.
• Design human QC mechanisms and calibration processes.
• Run full product lifecycle from discovery through post-launch audits.

Responsibilities

• Define core quality metrics like first-pass-right and coverage.
• Design the human QC mechanism and calibration process between automated judges and human raters.
• Run the full product lifecycle from discovery through post-launch audit with written hypotheses.

Job description

• You will own the Evals Quality charter, defining success metrics and improving creative quality across Pencil's agentic system.
• You will build and maintain the judging methodology, rubric design, and golden dataset to ensure reliable measurement.
• You will partner with commercial teams to provide evidence for client RFPs and enablement packages.

Key Responsibilities:

• Define core quality metrics like first-pass-right and coverage, and drive them up.
• Design the human QC mechanism and calibration process between automated judges and human raters.
• Run the full product lifecycle from discovery through post-launch audit with written hypotheses.

What We Offer:

• 25 days PTO plus public holidays, with a flexible time off scheme.
• Health insurance, wellness stipend, and enhanced parental leave policies.
• Fully remote role with flexible hours and access to London office if desired.

Pencil is an advertising technology company using Generative AI to redefine content creation for brands. Founded in 2018 by former Google, Facebook and Uber employees, it is backed by Sequoia and now part of The Brandtech Group, fostering a fast-moving, human-centric culture.