LLM Evaluations Engineer — Remote Benchmarking & Tools

Jaide Health; United States; On-site; USD 90,000 - 130,000; Full time

Benefits offered by this job

Fully remote work & flexible hours

37 days/year of vacation & holidays

Health insurance allowance for you and dependents

Company-provided equipment

Wellbeing and home office allowances

Frequent team get togethers

Diverse & inclusive people-first culture

Job summary

Jaide Health is looking for a full-time remote employee who will design and implement infrastructure used by researchers and engineers. Your mission is to ensure progress leads to meaningful improvements for users. Responsibilities include collaborating on model evaluations, defining metrics, and working closely with peers. Ideal candidates will have experience in Large Language Models, strong programming skills, and a curious mindset. The role offers flexible hours and generous vacation days.

Qualifications

• Strong understanding and intuition of LLMs and their limitations.
• Good taste and a curious mindset.
• Use modern tools and always looking to improve.

Responsibilities

• Research and implement evaluations and benchmarks for both base and instruction following models.
• Collaborate to define meaningful metrics and evaluations.
• Work in team, plan future steps, and communicate clearly.

Skills

Experience with Large Language Models (LLM)

Programming experience in multiple languages

Strong algorithmic skills

Strong engineering background

Familiar with full software development life cycle

Critical thinking

Tools

Linux

Python

Job description

Jaide Health is looking for a full-time remote employee who will design and implement infrastructure used by researchers and engineers. Your mission is to ensure progress leads to meaningful improvements for users. Responsibilities include collaborating on model evaluations, defining metrics, and working closely with peers. Ideal candidates will have experience in Large Language Models, strong programming skills, and a curious mindset. The role offers flexible hours and generous vacation days.