Remote AI Scientist (Physics) — Task Designer & Evaluator

Planet Pharma; San Francisco (CA); Remote; USD 120,000 - 180,000; Full time

Job summary

Planet Pharma seeks experienced scientists and engineers to evaluate frontier AI models on real technical work, including data analysis, design verification, experiments, and simulations. You will design challenging tasks drawn from your practice, with assumptions and checks, then run them through frontier AI agents and assess results to professional standards.

The role emphasizes self-directed, long-form work in a fully remote setting, with explicit focus on writing clear reasoning and robust

Qualifications

• Bachelor’s degree or higher; ideally 2+ years outside undergrad.
• 2+ years of applied experience in core sciences or engineering or data science.
• Comfortable writing and verifying work in code (Python or similar) and using the command line.
• Hands-on with real data and your field’s tools (MATLAB, Python, R; SPICE; FEA/CFD; GIS; spectroscopy software).
• Strong statistical and experimental reasoning; understanding of experimental design and uncertainty.

Responsibilities

• Design realistic technical tasks drawn from day-to-day work, with clear brief and supporting files.
• Run tasks through frontier AI models and evaluate outputs against professional standards.
• Compare model outputs on identical prompts and document performance gaps.
• Write detailed grading rubrics and explain pass/fail criteria in writing.
• Flag concrete failures with evidence and ensure proper interpretation of the ask.
• Collaborate across disciplines to refine tasks and reviewer processes.

Skills

Python

Data analysis

Experiment design

Statistical reasoning

Cloud tools

Education

Bachelor's degree or higher

Tools

Python

MATLAB

R

SPICE

FEA/CFD

GIS

Chromatography software

Spectroscopy software

PCB tools

Job description

Planet Pharma seeks experienced scientists and engineers to evaluate frontier AI models on real technical work, including data analysis, design verification, experiments, and simulations. You will design challenging tasks drawn from your practice, with assumptions and checks, then run them through frontier AI agents and assess results to professional standards.

The role emphasizes self-directed, long-form work in a fully remote setting, with explicit focus on writing clear reasoning and robust