Remote Senior LLM Evaluation Engineer

Binance

United States

Remote

USD 120,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Remote-first

Job summary

Binance is seeking an Evaluation Expert with a strong algorithmic background to build comprehensive LLM evaluation capabilities across dialogue and trading scenarios. You will quantify model performance, identify issues, and guide R&D directions with metrics, rubrics, and robust data pipelines.

Responsibilities include designing end-to-end evaluation plans, building benchmarks, and driving automation to support rapid model iteration in a global, remote-first environment.

Qualifications

  • Master's degree or above in Computer Science, Artificial Intelligence, Mathematics, Statistics, or related fields.

Responsibilities

  • Design end-to-end LLM evaluation plans for dialogue and financial trading scenarios.
  • Build evaluation metric systems and rubrics to quantify model performance.
  • Lead the construction of evaluation datasets and ensure data quality guidelines are followed.
  • Define evaluation dimensions, coverage, and create reliable benchmarks reflected in business needs.
  • Analyze model capabilities and failure modes to drive RD iteration and improvements.
  • Drive automation and scaling of evaluation workflows and toolchains.

Skills

Python
LLM evaluation
Communication skills
Data processing

Education

Master's degree or above in Computer Science/AI/Math/Statistics

Job description

Binance is seeking an Evaluation Expert with a strong algorithmic background to build comprehensive LLM evaluation capabilities across dialogue and trading scenarios. You will quantify model performance, identify issues, and guide R&D directions with metrics, rubrics, and robust data pipelines.

Responsibilities include designing end-to-end evaluation plans, building benchmarks, and driving automation to support rapid model iteration in a global, remote-first environment.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote-First Senior LLM Evaluation Architect
Remote-First Senior LLM Evaluation Architect

Remote • San Francisco (CA)

Remote
USD 150,000 - 210,000
Remote-first
Competitive salary
Global career growth
+1
Senior Evaluation Algorithm Engineer(APAC - Remote)
Senior Evaluation Algorithm Engineer(APAC - Remote)

Binance • United States

Remote
USD 120,000 - 180,000
Remote-first
Senior Evaluation Algorithm Engineer(APAC - Remote)
Senior Evaluation Algorithm Engineer(APAC - Remote)

Remote • San Francisco (CA)

Remote
USD 150,000 - 210,000
Remote-first
Competitive salary
Global career growth
+1
Member of Technical Staff, Model Evaluation
Member of Technical Staff, Model Evaluation

Inception Labs • Palo Alto (CA)

Hybrid
USD 150,000 - 185,000
Remote LLM Data Scientist & AI Systems Engineer
Remote LLM Data Scientist & AI Systems Engineer

Binance • United States

Remote
AUD 150,000 - 230,000
Remote-first work culture
Remote Senior Ruby Engineer — LLM Evaluation & Validation
Remote Senior Ruby Engineer — LLM Evaluation & Validation

Turing • United States

Remote
USD 83,000 - 124,000
Fully remote environment
Cutting-edge AI projects
LLM Data Scientist/Algorithm Engineer (Top Talent Program) (Fully Remote)
LLM Data Scientist/Algorithm Engineer (Top Talent Program) (Fully Remote)

Binance • United States

Remote
USD 140,000 - 190,000
Senior Research Scientist, Model Evaluation (Remote‑Flexible)
Senior Research Scientist, Model Evaluation (Remote‑Flexible)

Cohere • New York (NY)

On-site
USD 140,000 - 200,000
Open and inclusive culture
Work on cutting-edge AI research
Weekly lunches and snacks
+6
Remote LLM Data Scientist & AI Systems Engineer
Remote LLM Data Scientist & AI Systems Engineer

Binance • United States

Remote
USD 140,000 - 190,000
Senior Software Engineer - Remote LLM Eval
Senior Software Engineer - Remote LLM Eval

Turing • United States

Remote
USD 41,000 - 83,000
Fully remote