Remote LLM Evaluation Engineer

Veeva Systems, Inc.; Ottawa; Hybrid; CAD 85,000 - 225,000; Full time

Job summary

A leading life sciences technology company seeks a candidate for a crucial role focused on ensuring the reliability and safety of AI Agents. Responsibilities include defining evaluation strategies, assessing AI outputs, and developing automated evaluation pipelines. Candidates should have a Bachelor's in Data Science or similar field and proficiency in Python. The role supports a flexible work environment. Competitive compensation ranges from $85,000 to $225,000 CAD, based on qualifications and experience.

Benefits

Medical, dental, vision, and basic life insurance

PTO and company-paid holidays

Retirement programs

1% charitable giving program

Qualifications

• Strong understanding of data quality principles and validation methods.
• Experience with prompt engineering and AI evaluation.
• Proficiency in designing automated evaluation pipelines.

Responsibilities

• Define comprehensive evaluation strategies for AI Agents.
• Evaluate quality of LLM-generated content.
• Design and curate high-quality test datasets.
• Develop scalable automated evaluations.

Skills

Analytical capabilities

Problem-solving skills

Communication

Quick learning

Education

Bachelor’s degree in Data Science, Machine Learning, Computer Science

Tools

Python

CI/CD systems

Job description

A leading life sciences technology company seeks a candidate for a crucial role focused on ensuring the reliability and safety of AI Agents. Responsibilities include defining evaluation strategies, assessing AI outputs, and developing automated evaluation pipelines. Candidates should have a Bachelor's in Data Science or similar field and proficiency in Python. The role supports a flexible work environment. Competitive compensation ranges from $85,000 to $225,000 CAD, based on qualifications and experience.