Data Analysis & Evaluation Engineer (Remote)

Cohere; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

Cohere is seeking a Member of Technical Staff in Data Analysis and Evaluation to ensure the quality, reliability, and performance of our large language models. You will design data collection tasks, assess dataset quality, and analyse robustness across diverse scenarios, collaborating with researchers, engineers, and data annotators.

This role combines statistics, experimental design with human annotators, and machine learning to train and evaluate models on high-quality data.

Qualifications

• Strong software engineering skills.
• Experience designing data collection tasks with human annotators.
• Strong statistical skills and experimental design.
• Experience evaluating datasets for quality and biases.

Responsibilities

• Design and oversee data collection tasks, including supporting human annotators and ensuring data quality.
• Develop and apply statistical methods to evaluate the quality and reliability of datasets.
• Analyse and assess the generalisability and robustness of ML systems across diverse use cases.
• Collaborate with teams to improve dataset quality and model performance.
• Train and fine-tune large language models (LLMs) on distributed training infrastructures.
• Conduct experiments to evaluate model performance and identify areas for improvement.

Skills

Software engineering

Data analysis

Statistics

Experimental design

ML basics

Collaboration

Tools

Python

SQL

Pandas

SciPy

Job description

Cohere is seeking a Member of Technical Staff in Data Analysis and Evaluation to ensure the quality, reliability, and performance of our large language models. You will design data collection tasks, assess dataset quality, and analyse robustness across diverse scenarios, collaborating with researchers, engineers, and data annotators.

This role combines statistics, experimental design with human annotators, and machine learning to train and evaluate models on high-quality data.