Research Scientist, Benchmarks & Evaluations for Frontier AI

Kled AI; California City (CA); On-site; USD 150,000 - 250,000; Full time

Job summary

Kled AI is seeking a Research Scientist for Benchmarks & Eval to shape how frontier AI models are evaluated using real opt-in data.

You will design public benchmarks, create private test sets, run full end-to-end experiments from GPUs to reports, and publish trusted leaderboards that help researchers and buyers understand data value. Collaboration with founders and lab buyers accelerates data-driven deals.

Qualifications

• 3+ years ML research or research engineering experience.
• A benchmark or evaluation you built that others actually used.
• Strong hands-on experience training and fine-tuning image or video models (PyTorch).
• Deep understanding of experimental design and statistics (controls, ablations, human studies).
• Experience running experiments end to end, from GPUs to final report.
• Clear writing for both researchers and buyers.

Responsibilities

• Design and ship public benchmarks for image, video and document models.
• Build private test sets from real, opt-in data that no model has trained on.
• Run data-value experiments: train models with and without Kled data, measure changes.
• Design human evaluation studies and the automatic metrics that track them.
• Publish leaderboards and reports that researchers at frontier labs trust.
• Find where models fall short and tell our team what data to collect next.
• Work directly with the founders and lab buyers to turn results into data deals.

Skills

ML research

Benchmark design

PyTorch training

Experimental design

End-to-end experiments

Technical writing

Tools

PyTorch

Job description

Kled AI is seeking a Research Scientist for Benchmarks & Eval to shape how frontier AI models are evaluated using real opt-in data.

You will design public benchmarks, create private test sets, run full end-to-end experiments from GPUs to reports, and publish trusted leaderboards that help researchers and buyers understand data value. Collaboration with founders and lab buyers accelerates data-driven deals.