Staff, Model Evaluations & Behavioral Metrics (Equity)

Simile; San Francisco (CA); On-site; USD 200,000 - 400,000; Full time

Benefits offered by this job

Health & Wellness benefits

Flexible time off

Equity grants

Job summary

Simile is seeking a Member of Technical Staff, Model Evaluations, to develop measurement systems that gauge the accuracy of simulations of human behavior. You will design evaluation metrics and collaborate with modeling teams to ensure models are effective.

Ideal candidates possess strong statistical judgement, familiarity with LLMs, and the ability to automate evaluation workflows. The role offers a salary range of $200,000 – $400,000 USD along with equity and comprehensive benefits.

Qualifications

• Strong statistical and evaluation intuition.
• Ability to build internal tools and automate workflows.
• Comfortable with noisy data and making grounded decisions.

Responsibilities

• Design evaluations and metrics for behavior simulation.
• Evaluate model performance and suggest improvements.
• Automate workflows and create internal tools.

Skills

Evaluation Taste

LLM and Model Fluency

Statistical Judgment

Technical and Agentic Execution

Hands-On Ownership

Education

Experience in LLM evals, applied ML research, data science or related fields

Tools

Python

SQL

R

Job description

Simile is seeking a Member of Technical Staff, Model Evaluations, to develop measurement systems that gauge the accuracy of simulations of human behavior. You will design evaluation metrics and collaborate with modeling teams to ensure models are effective.

Ideal candidates possess strong statistical judgement, familiarity with LLMs, and the ability to automate evaluation workflows. The role offers a salary range of $200,000 – $400,000 USD along with equity and comprehensive benefits.