Foundation Models Evaluation Scientist (Multimodal ML)

Apple; Boulder (CO); On-site; USD 132,100 - 244,600; Full time

Benefits offered by this job

Comprehensive medical and dental coverage

Retirement benefits

Educational reimbursement

Employee stock purchase plan

Job summary

A leading technology firm is seeking an expert in evaluating machine learning and deep learning models. You will develop robust methodologies to assess foundation models and collaborate with teams to ensure product quality. The ideal candidate has strong analytical thinking, expertise in Python, and experience in evaluating LLMs and traditional statistical methods. This is a critical role for crafting evaluation frameworks and improving model robustness. Competitive compensation and benefits are offered including comprehensive medical and dental coverage.

Qualifications

• Minimum of 3 years relevant industry experience.
• Strong experience in evaluating supervised, unsupervised, and deep learning models.
• Proficiency in Python and familiarity with multilayer models.

Responsibilities

• Develop robust methodologies to assess foundation models.
• Build and curate evaluation datasets and benchmarks.
• Conduct failure analysis to improve model robustness.

Skills

Analytical thinking

Expertise in Python

Statistical methodologies

Hands-on experience evaluating LLMs

Strong documentation skills

Education

BS degree

Tools

NumPy

pandas

scikit-learn

PyTorch

TensorFlow

Job description

A leading technology firm is seeking an expert in evaluating machine learning and deep learning models. You will develop robust methodologies to assess foundation models and collaborate with teams to ensure product quality. The ideal candidate has strong analytical thinking, expertise in Python, and experience in evaluating LLMs and traditional statistical methods. This is a critical role for crafting evaluation frameworks and improving model robustness. Competitive compensation and benefits are offered including comprehensive medical and dental coverage.