Staff Engineer, Evaluation Infrastructure

Simile; San Francisco (CA); On-site; USD 200,000 - 400,000; Full time

Benefits offered by this job

Health & Wellness

Equity

Flexible time off

Job summary

Simile is building the first AI simulation of society and is hiring for a Member of Technical Staff in Evaluations Engineering. You will create scalable systems to evaluate how well simulations match ground truth, with influence across data, backend services, automation, and internal tooling.

You will partner with Evals, Modeling, Product Engineering, and Data Operations. You will streamline evaluation runs, strengthen versioning and access controls, automate customer validations, and develop

Qualifications

• Strong engineering fundamentals for production-quality software.
• Experience building backend services, data pipelines, automation workflows, and relational data models.
• Ability to take ambiguous projects from design through deployment and adoption.
• Strong intuition for reliable evaluation infrastructure including versioning and reproducibility.
• Familiarity with modern model-development and evaluation workflows.
• Ability to build clear tools for researchers, engineers, and operators.
• Proven track record of independently driving technical work across teams.

Responsibilities

• Build evaluation execution infrastructure: pipelines and orchestration across datasets, models, populations, use cases.
• Strengthen evaluation data systems: design relational schemas, versioning, provenance, permissions, quality controls.
• Automate validation and data collection: streamline validations, survey deployment, response ingestion, ground truth integration.
• Build human data workflows: create labeling and review tools for external experts and operators.
• Develop evaluation tooling: interfaces to manage evals, compare models, investigate results, and identify regressions.

Skills

Engineering fundamentals

Data pipelines

End-to-end execution

Evaluation judgment

ML/LLM fluency

Product & user judgment

Ownership & communication

Job description

Simile is building the first AI simulation of society and is hiring for a Member of Technical Staff in Evaluations Engineering. You will create scalable systems to evaluate how well simulations match ground truth, with influence across data, backend services, automation, and internal tooling.

You will partner with Evals, Modeling, Product Engineering, and Data Operations. You will streamline evaluation runs, strengthen versioning and access controls, automate customer validations, and develop