GenAI Trust & Evaluation Engineer

Equinix

Toronto

On-site

CAD 70,000 - 110,000

Full time

7 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Equinix in Toronto designs, develops and deploys evaluation and trust capabilities for our GenAI platform. You will build platform features for evaluation datasets, golden data, synthetic tests and scoring with LLM-as-judge, ensuring scalable and secure agent assessments.

Collaborate with AI engineers, product, platform and security teams to fix failures, improve guardrails, and monitor agent quality in production. Experience with logs/traces and evaluation statistics is valued.

Qualifications

  • 1–4 years in AI/ML or software engineering.
  • Bachelor's degree in computer science, engineering, data science, mathematics or a related field, or equivalent experience
  • Proficiency in Python, with solid habits: version control, testing, code review. Experience building backend services or platform features
  • Hands-on experience building, not only using, evaluations or test systems for ML or LLM applications, including reading failures from logs or traces
  • Working knowledge of LLM applications (prompting, retrieval, agents)
  • Working knowledge of evaluation statistics (sampling, confidence intervals, variance across runs) and of measuring agreement between reviewers
  • Ability to write SQL and analyze results
  • Clear written and verbal communication

Responsibilities

  • Evaluation platform (primary focus)
  • Build platform features for evaluation: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
  • Build scoring for agents: code-based checks, LLM-as-judge, tool-call accuracy and grounding checks
  • Build human-review features that compare judge scores against human reviewers, and recalibrate when agreement is low
  • Run evaluations at scale: before release, on every change to a prompt, model or tool, and on agents in production
  • Build evaluation playbooks so agent builders can evaluate their agents without specialist help
  • Keep results traceable to the agent, model, prompt and dataset versions tested, with a record of who approved each release
  • Trust and safety
  • Build and test the platform's trust controls: agent identity, per-user access to connected systems, tool permissions, guardrails and data boundaries
  • Run adversarial tests and red-teaming (prompt injection, out-of-scope requests, data-boundary tests), and turn findings into fixes and regression tests
  • Work with security and governance teams to turn policy requirements into automated checks
  • Root-cause analysis and quality
  • Investigate agent failures and performance problems end to end, and follow them until the cause is fixed
  • Build test and performance tooling: end-to-end agent tests, load and latency tests, and failure diagnostics
  • Turn production failures and user feedback into regression tests, so the same defect does not return
  • Quality and responsible practice
  • Write tested, reviewed and documented code, and contribute to shared libraries and tooling
  • Test and build to security, data-privacy and responsible-AI requirements

Skills

Python programming
Analytical skills
Clear communication
SQL querying
Log analysis
LLM knowledge

Education

Bachelor's degree in computer science, engineering, data science, mathematics or a related field, or equivalent experience

Tools

Python
SQL

Job description

Equinix in Toronto designs, develops and deploys evaluation and trust capabilities for our GenAI platform. You will build platform features for evaluation datasets, golden data, synthetic tests and scoring with LLM-as-judge, ensuring scalable and secure agent assessments.

Collaborate with AI engineers, product, platform and security teams to fix failures, improve guardrails, and monitor agent quality in production. Experience with logs/traces and evaluation statistics is valued.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Engineer, Trust & Evaluation
AI Engineer, Trust & Evaluation

Equinix • Toronto

On-site
CAD 70,000 - 110,000
Senior AI Enablement Architect & LLM Expert
Senior AI Enablement Architect & LLM Expert

Equinix • Toronto

On-site
CAD 142,000 - 212,000
AI EVAL/QE — Toron
AI EVAL/QE — Toron

Best Your Buy • Toronto

Hybrid
CAD 120,000 - 180,000
AI Experience Engineer: Builder Tools & Conversational UX
AI Experience Engineer: Builder Tools & Conversational UX

Equinix • Toronto

On-site
CAD 102,000 - 152,000
Agentic AI Optimization Developer
Agentic AI Optimization Developer

Equifax Canada • Toronto

On-site
CAD 110,000 - 150,000
Senior GenAI Engineer: Prototyping to Production
Senior GenAI Engineer: Prototyping to Production

Equinix • Toronto

On-site
CAD 142,000 - 212,000
Senior AI Engineer - Toronto, ON
Senior AI Engineer - Toronto, ON

Pozent Corporation • Toronto

On-site
CAD 110,000 - 170,000
Production Reliability Engineer, AI Platform
Production Reliability Engineer, AI Platform

Equinix • Toronto

On-site
CAD 102,000 - 152,000
Employee Assistance Program
Principal Product Manager, Agent Intelligence
Principal Product Manager, Agent Intelligence

Equinix • Toronto

On-site
CAD 158,000 - 236,000
AI Engineer
AI Engineer

Armilla AI • Toronto

On-site
CAD 90,000 - 130,000