Senior Technical Staff, AI Safety for Agents (Remote)

Cohere Inc.; Toronto; Remote; CAD 140,000 - 190,000; Full time

Benefits offered by this job

Lunch stipend

Health and dental benefits

RRSP matching / Pension

Parental leave top-up

Education stipend and conferences

Vacation (6 weeks)

Job summary

Cohere Inc. is hiring for a Member of Technical Staff in the Safety for Agents team.

You will impact the development of safer, more trustworthy LLMs, focusing on data generation, post-training algorithms, and evaluation methods to ensure safety as models access external resources. You will work with cross-functional ML teams and data annotation teams, collaborating with product and policy teams, applying machine learning, ethical AI, and experimental design to advance the next generation of

Qualifications

• PhD/MS in a relevant field is a plus but not required.
• Experience evaluating experiments and data collection pipelines is essential.
• Strong software engineering practices and test coverage are expected.
• Proven ability to work with cross-functional teams and document findings.

Responsibilities

• Lead data generation and post-training evaluation for Safety in LLMs.
• Design and oversee data collection tasks, including annotator collaboration.
• Analyze datasets for quality, biases, and suitability for training ML models.
• Train and fine-tune LLMs on distributed training infrastructures.
• Assess generalizability and robustness of ML systems.
• Communicate findings effectively to product and policy teams.

Skills

Statistical analysis

Software engineering

Data collection design

Dataset bias evaluation

LLM training

ML robustness evaluation

Python ML frameworks

Communication

Research publications

Tools

PyTorch

TensorFlow

JAX

Job description

Cohere Inc. is hiring for a Member of Technical Staff in the Safety for Agents team.

You will impact the development of safer, more trustworthy LLMs, focusing on data generation, post-training algorithms, and evaluation methods to ensure safety as models access external resources. You will work with cross-functional ML teams and data annotation teams, collaborating with product and policy teams, applying machine learning, ethical AI, and experimental design to advance the next generation of