ML Research Engineer: Fine-Tuning & Evaluation Systems

HonestAI; San Francisco (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

HonestAI in San Francisco is seeking a researcher to join a small team focused on fine-tuning, retrieval, and agent evaluation. You will bridge papers to production and run controlled experiments on agent quality and cost.

You will build evaluation harnesses and offline benchmarks, publish internal notes that inform product decisions, and collaborate with ML researchers and engineers to advance model performance and reliability.

Qualifications

• MS/PhD in CS/ML or related, or equivalent experience.
• Proven track record shipping evaluation or fine-tuning systems.
• Fluent in PyTorch.

Responsibilities

• Run controlled experiments on agent quality and cost.
• Build evaluation harnesses and offline benchmarks.
• Publish internal research notes that drive product decisions.

Skills

PyTorch

Evaluation

Education

MS / PhD in CS / ML or related

Job description

HonestAI in San Francisco is seeking a researcher to join a small team focused on fine-tuning, retrieval, and agent evaluation. You will bridge papers to production and run controlled experiments on agent quality and cost.

You will build evaluation harnesses and offline benchmarks, publish internal notes that inform product decisions, and collaborate with ML researchers and engineers to advance model performance and reliability.