ML Evaluation Engineer: Benchmark & Model Quality

Reducto; San Francisco (CA); On-site; USD 100,000 - 130,000; Full time

Benefits offered by this job

Unlimited PTO

Daily free lunch

Reimbursed transportation

Generous health insurance

Health and wellness budget

Flexible parental leave

Job summary

A leading AI solutions company in San Francisco is seeking an ML Eval Engineer to design evaluation benchmarks and improve model performance. This role involves working with unstructured enterprise data and collaborating closely with the ML and engineering teams. You will develop metrics, conduct evaluations, and contribute to model enhancements in a fast-paced environment. If you enjoy solving complex problems and care about precision, this is the role for you.

Qualifications

• Strong Python skills to build and maintain technical solutions.
• Experience with data infrastructure, particularly AWS S3.
• Ability to work with unstructured enterprise data.

Responsibilities

• Design and maintain evaluation benchmarks for model performance.
• Develop metrics and workflows to identify failure modes.
• Collaborate with ML engineers to improve model quality.

Skills

Python

Data analysis

Model evaluation

Collaboration

Tools

AWS S3

OLAP systems

Flask

Job description

A leading AI solutions company in San Francisco is seeking an ML Eval Engineer to design evaluation benchmarks and improve model performance. This role involves working with unstructured enterprise data and collaborating closely with the ML and engineering teams. You will develop metrics, conduct evaluations, and contribute to model enhancements in a fast-paced environment. If you enjoy solving complex problems and care about precision, this is the role for you.