Senior LLM Evaluation Engineer — Metrics & Automation

Binance; Hong Kong; On-site; HKD 600,000 - 1,000,000; Full time

Benefits offered by this job

Work-from-home arrangement

Competitive salary and benefits

Career development opportunities

Job summary

Binance is seeking an evaluation expert with an algorithmic background to build LLM evaluation capabilities spanning dialogue, trading, and other business scenarios. You will quantify model performance, pinpoint issues, and drive continuous model improvement within a fast-paced, global fintech environment.

The role emphasizes designing robust evaluation plans, constructing high-quality datasets, and scaling evaluation workflows to support rapid model iterations while collaborating with multiple

Qualifications

• Master's degree or above in CS/AI/Math/Stats with solid algorithmic foundation.
• Hands-on LLM evaluation experience at a large tech company involved in commercial deployment.
• Familiar with evaluation methods and able to define dimensions for different business scenarios.
• Systematic control over evaluation data representativeness, annotation consistency, and result reliability.
• Proficient in Python and automation of evaluation workflows.
• Strong business understanding and cross-team communication.

Responsibilities

• Design end-to-end LLM evaluation plans for dialogue and financial trading scenarios.
• Lead dataset construction with clear rubrics and quality control processes.
• Analyze results to identify failures and provide actionable improvement recommendations.
• Develop scalable evaluation platforms and toolchains for rapid model iteration.
• Collaborate with algorithm, product, and data teams to translate objectives into evaluation standards and drive RD directions.

Skills

Python

LLM evaluation

Data analysis

Cross-functional collaboration

Communication

Education

Master's degree in CS/AI/Math/Stats

Tools

Evaluation platforms

Benchmark construction

Python automation

Job description

Binance is seeking an evaluation expert with an algorithmic background to build LLM evaluation capabilities spanning dialogue, trading, and other business scenarios. You will quantify model performance, pinpoint issues, and drive continuous model improvement within a fast-paced, global fintech environment.

The role emphasizes designing robust evaluation plans, constructing high-quality datasets, and scaling evaluation workflows to support rapid model iterations while collaborating with multiple