Senior Staff AI Platform Engineer — GenAI & ML Evaluation

Pinterest; San Francisco (CA); Hybride; USD 209 000 - 429 000; Plein temps

Avantages offerts par ce poste

Equity

Résumé du poste

Pinterest is seeking a Senior Staff Software Engineer for the AI Platform to shape and scale our AI/ML evaluation systems company-wide. You will identify modernization opportunities, elevate the roadmap, and execute a durable evaluation strategy for GenAI and ML workloads.

Collaborating with engineering, research, product, and safety teams, you will drive cross-team adoption of shared evaluation capabilities, make durable architectural decisions, and balance automated metrics with human judgment

Qualifications

• 3+ years designing and operating GenAI/ML evaluation systems with benchmarks and golden datasets.
• Proficient in Python and distributed systems; comfortable using AI tools for coding/design.
• Experience with GenAI evaluation libraries such as AutoEvals, OpenEvals and Ragas.
• 5+ years building scalable platforms for distributed experimentation, versioning, observability, and release management.

Responsabilités

• Identify modernization opportunities and scale the roadmap for a company-wide evaluation system.
• Define cross-team technical strategy and drive broad adoption of shared evaluation capabilities.
• Make durable architectural decisions to improve user experience and reliability of the AI/ML evaluation platform.
• Establish evaluation methodologies balancing automated metrics, expert judgment, human feedback, safety, and business outcomes.
• Lead complex cross-team workstreams, resolve technical tradeoffs and risks, and deliver impact.

Connaissances

Python

Distributed systems

GenAI evaluation

Recommender systems

Model versioning

Observability

Release management

Production monitoring

Formation

Bachelor's degree or equivalent

Outils

AutoEvals

OpenEvals

Ragas

Description du poste

Pinterest is seeking a Senior Staff Software Engineer for the AI Platform to shape and scale our AI/ML evaluation systems company-wide. You will identify modernization opportunities, elevate the roadmap, and execute a durable evaluation strategy for GenAI and ML workloads.

Collaborating with engineering, research, product, and safety teams, you will drive cross-team adoption of shared evaluation capabilities, make durable architectural decisions, and balance automated metrics with human judgment