Staff Engineer, Model Evaluations & Behavioral Metrics

Simile; New York, Northern (NY, KY); Hybride; USD 200 000 - 400 000; Plein temps

Avantages offerts par ce poste

Health & Wellness

Time Off

Equity grants

Résumé du poste

Simile is hiring a Member of Technical Staff, Model Evaluations, to build the measurement layer for behavioral simulations and ensure simulations align with real human data. You will work on evals across distributions, ground truth, and uncertainty, supporting product decisions with rigorous metrics.

You will collaborate with modeling, research, and engineering teams to maintain robust evaluation pipelines, dashboards, and labeling tooling while driving a fast, rigorous research culture.

Qualifications

• Evaluation-focused mindset with ability to explain what an eval measures and its limitations.
• Understanding of modern LLM training, evaluation, and model behavior.
• Ability to reason about data, uncertainty, and measurement validity.
• Capability to build tools and dashboards for evaluation pipelines.
• Willingness to drive a project from start to finish.

Responsabilités

• Build the measurement layer for behavioral simulation with dashboards and workflows.
• Evaluate new model versions and maintain stable eval suites.
• Create evals for qualitative responses, surveys, and product surfaces.
• Compare simulated outcomes to human data and reason about ground truth.
• Automate evaluation workflows and develop labeling processes.
• Prototype diverse data sources to advance behavioral predictions.

Connaissances

Evaluation taste

LLM fluency

Statistical judgment

Technical execution

Hands-on ownership

Description du poste

Simile is hiring a Member of Technical Staff, Model Evaluations, to build the measurement layer for behavioral simulations and ensure simulations align with real human data. You will work on evals across distributions, ground truth, and uncertainty, supporting product decisions with rigorous metrics.

You will collaborate with modeling, research, and engineering teams to maintain robust evaluation pipelines, dashboards, and labeling tooling while driving a fast, rigorous research culture.