AI Engineer, Trust & Evaluation

Equinix

Toronto

Presencial

CAD 70.000 - 110.000

Jornada completa

Hace 7 días
Sé de los primeros/as/es en solicitar esta vacante

Una candidatura completa en un minuto

Currículum y carta de presentación personalizados, listos para enviar.

Supera los filtros ATS

Descripción de la vacante

Equinix in Toronto designs, develops and deploys evaluation and trust capabilities for our GenAI platform. You will build platform features for evaluation datasets, golden data, synthetic tests and scoring with LLM-as-judge, ensuring scalable and secure agent assessments.

Collaborate with AI engineers, product, platform and security teams to fix failures, improve guardrails, and monitor agent quality in production. Experience with logs/traces and evaluation statistics is valued.

Formación

  • 1–4 years in AI/ML or software engineering.
  • Bachelor's degree in computer science, engineering, data science, mathematics or a related field, or equivalent experience
  • Proficiency in Python, with solid habits: version control, testing, code review. Experience building backend services or platform features
  • Hands-on experience building, not only using, evaluations or test systems for ML or LLM applications, including reading failures from logs or traces
  • Working knowledge of LLM applications (prompting, retrieval, agents)
  • Working knowledge of evaluation statistics (sampling, confidence intervals, variance across runs) and of measuring agreement between reviewers
  • Ability to write SQL and analyze results
  • Clear written and verbal communication

Responsabilidades

  • Evaluation platform (primary focus)
  • Build platform features for evaluation: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
  • Build scoring for agents: code-based checks, LLM-as-judge, tool-call accuracy and grounding checks
  • Build human-review features that compare judge scores against human reviewers, and recalibrate when agreement is low
  • Run evaluations at scale: before release, on every change to a prompt, model or tool, and on agents in production
  • Build evaluation playbooks so agent builders can evaluate their agents without specialist help
  • Keep results traceable to the agent, model, prompt and dataset versions tested, with a record of who approved each release
  • Trust and safety
  • Build and test the platform's trust controls: agent identity, per-user access to connected systems, tool permissions, guardrails and data boundaries
  • Run adversarial tests and red-teaming (prompt injection, out-of-scope requests, data-boundary tests), and turn findings into fixes and regression tests
  • Work with security and governance teams to turn policy requirements into automated checks
  • Root-cause analysis and quality
  • Investigate agent failures and performance problems end to end, and follow them until the cause is fixed
  • Build test and performance tooling: end-to-end agent tests, load and latency tests, and failure diagnostics
  • Turn production failures and user feedback into regression tests, so the same defect does not return
  • Quality and responsible practice
  • Write tested, reviewed and documented code, and contribute to shared libraries and tooling
  • Test and build to security, data-privacy and responsible-AI requirements

Conocimientos

Python programming
Analytical skills
Clear communication
SQL querying
Log analysis
LLM knowledge

Educación

Bachelor's degree in computer science, engineering, data science, mathematics or a related field, or equivalent experience

Herramientas

Python
SQL

Descripción del empleo

Who are we?

Equinix is the world’s digital infrastructure company®, shortening the path to connectivity to enable the innovations that enrich our work, life and planet.

A place where tech thinkers and future builders turn bold ideas into breakthrough experiences, we welcome your unique perspective. Help us challenge assumptions, uncover bias, and remove barriers—because progress starts with fresh ideas. You’ll find belonging, purpose, and a team that welcomes you—because when you feel valued, you’re empowered to do your best work.

Job Summary

Designs, develops and deploys the evaluation and trust capabilities of Equinix's GenAI platform. Builds platform features that let teams evaluate their agents at scale: golden datasets, synthetic test data, scoring with LLM-as-judge, and monitoring of agent quality in production. Builds and tests the trust controls that agents run on, including agent identity, access, guardrails and data boundaries. Traces agent failures to their root cause and fixes them in the platform. Works in a team with senior engineers and with product, platform and security partners to turn defined problems into robust, secure, observable and performant solutions.

Knowledge and Application

Applies solid knowledge of software engineering practice and of LLM and agent concepts (prompting, retrieval, tool and function calling, orchestration, MCP). Understands evaluation methods (code-based checks, LLM-as-judge, human review, golden datasets, regression suites) and evaluation statistics (sampling, variance across runs, agreement between reviewers). Understands AI security risks such as prompt injection and data leakage, and identity concepts such as OAuth and per-user access. Follows established standards and suggests improvements.

Complexity and Problem Solving

Works on problems of defined scope where analysis requires evaluation of identifiable factors. Traces an agent failure to its cause (retrieval, tool call, prompt, model, data or infrastructure) using logs and traces, and follows it until it is fixed. Judges whether a metric reflects real quality. Escalates unusual or high-impact issues. Work is reviewed at defined checkpoints.

Collaboration and Interaction

Works with AI engineers, product managers, forward-deployed engineers, platform and security teams. Turns the needs of agent builders into platform features. Explains results, defects and risks to partners who are not familiar with the subject. Asks for guidance early and shares knowledge with the team.

Responsibilities

Evaluation platform (primary focus)

  • Build platform features for evaluation: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
  • Build scoring for agents: code-based checks, LLM-as-judge, tool-call accuracy and grounding checks
  • Build human-review features that compare judge scores against human reviewers, and recalibrate when agreement is low
  • Run evaluations at scale: before release, on every change to a prompt, model or tool, and on agents in production
  • Build evaluation playbooks so agent builders can evaluate their agents without specialist help
  • Keep results traceable to the agent, model, prompt and dataset versions tested, with a record of who approved each release

Trust and safety

  • Build and test the platform's trust controls: agent identity, per-user access to connected systems, tool permissions, guardrails and data boundaries
  • Run adversarial tests and red-teaming (prompt injection, out-of-scope requests, data-boundary tests), and turn findings into fixes and regression tests
  • Work with security and governance teams to turn policy requirements into automated checks

Root-cause analysis and quality

  • Investigate agent failures and performance problems end to end, and follow them until the cause is fixed
  • Build test and performance tooling: end-to-end agent tests, load and latency tests, and failure diagnostics
  • Turn production failures and user feedback into regression tests, so the same defect does not return

Quality and responsible practice

  • Write tested, reviewed and documented code, and contribute to shared libraries and tooling
  • Test and build to security, data-privacy and responsible-AI requirements
Qualifications

Required

  • 1–4 years in AI/ML or software engineering (internships and applied research count)
  • Bachelor's degree in computer science, engineering, data science, mathematics or a related field, or equivalent experience
  • Proficiency in Python, with solid habits: version control, testing, code review. Experience building backend services or platform features
  • Hands-on experience building, not only using, evaluations or test systems for ML or LLM applications, including reading failures from logs or traces
  • Working knowledge of LLM applications (prompting, retrieval, agents)
  • Working knowledge of evaluation statistics (sampling, confidence intervals, variance across runs) and of measuring agreement between reviewers
  • Ability to write SQL and analyze results
  • Clear written and verbal communication
Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

AI Quality Engineer
AI Quality Engineer

Centraprise • Vancouver

Presencial
CAD 80.000 - 100.000
Principal AI Engineer (Context, Agents and Context)
Principal AI Engineer (Context, Agents and Context)

Elastic • Ottawa

Híbrido
CAD 140.000 - 210.000
Health coverage
Flexible location
Vacation days
+3
Agentic AI Optimization Developer
Agentic AI Optimization Developer

Equifax Canada • Toronto

Presencial
CAD 110.000 - 150.000
Principal Product Manager, Agent Intelligence
Principal Product Manager, Agent Intelligence

Equinix • Toronto

Presencial
CAD 158.000 - 236.000
Applied AI Engineer
Applied AI Engineer

Apptad • Montreal

Presencial
CAD 90.000 - 140.000
AI/Machine Learning Experience Engineer
AI/Machine Learning Experience Engineer

Equinix • Toronto

Presencial
CAD 102.000 - 152.000
Staff Engineer / AI Builder
Staff Engineer / AI Builder

F5 Remote Jobs • Canadá

A distancia
CAD 177.000 - 244.000
GenAI Trust & Evaluation Engineer
GenAI Trust & Evaluation Engineer

Equinix • Toronto

Presencial
CAD 70.000 - 110.000
Senior AI Engineer. Innovation and R&D
Senior AI Engineer. Innovation and R&D

Equinix • Toronto

Presencial
CAD 142.000 - 212.000
Staff AI Builder
Staff AI Builder

F5 Remote Jobs • Canadá

A distancia
CAD 177.000 - 244.000