Frontier AI Risk Evaluation Scientist

Scale; San Francisco, New York (CA, NY); On-site; USD 216,000 - 270,000; Full time

Benefits offered by this job

Health coverage

Retirement benefits

Learning stipend

Generous PTO

Commuter stipend

Job summary

Scale Labs seeks a Research Scientist focused on Frontier Risk Evaluations to design evaluation measures, harnesses and datasets for measuring risks posed by frontier AI systems. You will build harnesses to test models, collaborate with government agencies to scope evaluations, and publish methodologies for policymakers.

Ideal candidates have 3+ years in ML research or product development, a track record in generative AI, and strong cross-functional communication skills to work with industry,

Qualifications

• Commitment to advancing safe, secure and trustworthy AI deployments.
• Experience building ML pipelines, evaluation harnesses, and prototypes.
• Track record of published research in machine learning, especially generative AI.
• At least three years addressing sophisticated ML problems in research or product roles.
• Strong written and verbal communication to work cross-functionally.

Responsibilities

• Design and build evaluation measures, harnesses and datasets for frontier AI risk evaluation.
• Collaborate with government agencies or labs to scope and design risk assessments.
• Publish evaluation methodologies and write technical policy reports.

Skills

Research

ML pipelines

Evaluation harnesses

Generative AI

Communication

Tools

Python

ML libraries

Job description

Scale Labs seeks a Research Scientist focused on Frontier Risk Evaluations to design evaluation measures, harnesses and datasets for measuring risks posed by frontier AI systems. You will build harnesses to test models, collaborate with government agencies to scope evaluations, and publish methodologies for policymakers.

Ideal candidates have 3+ years in ML research or product development, a track record in generative AI, and strong cross-functional communication skills to work with industry,