AI, Data Scientist

PFF; Canada; Remote; CAD 90,000 - 120,000; Full time

Job summary

A leading sports analytics company in Canada is looking for an experienced AI Data Scientist to enhance LLM performance for various products including football chatbots and scouting assistants. The ideal candidate will have a strong background in applied AI/ML, possess excellent skills in Python, and have experience with transformer models and debugging tools. Join us in revolutionizing football data insights!

Qualifications

• 3+ years experience in applied AI/ML roles, with 1-2 years focused on LLMs.
• Deep understanding of transformer-based models and their limitations.
• Experience debugging LLM outputs using tools like LangSmith or W&B Traces.

Responsibilities

• Design robust evaluation frameworks for LLMs.
• Lead or assist with LLM distillation projects.
• Deploy dashboards and alerts to monitor model performance.

Skills

Applied AI/ML experience

Deep understanding of transformer models

Proficiency with Python

Experience with tracing/debugging

Strong communication skills

Tools

Hugging Face Transformers

LangChain

OpenAI API

PyTorch

TensorFlow

Job description

PFF is a leading sports analytics company that transforms complex football data into powerful insights. Our exclusive player grades, advanced metrics, and tools fuel winning decisions for NFL and college teams, broadcasters, fantasy players, bettors, and fans alike.

We are seeking an experienced AI Data Scientist with deep expertise in Large Language Models (LLMs) to lead efforts in model evaluation, tracing, optimization, and distillation. You will play a key role in enhancing the performance, reliability, and safety of generative AI systems across our consumer and B2B products including football chatbots, scouting assistants, and fan-facing analytics platforms.

What You’ll Do

-Design robust evaluation frameworks for LLMs (e.g., hallucination detection, factual accuracy, prompt robustness, calibration).

-Use model tracing tools (e.g., OpenAI trace logs, LangSmith, Weights & Biases) to analyze model behavior and failure modes.

-Lead or assist with LLM distillation projects, compressing large foundation models into smaller performant versions fine-tuned on football-specific domains.

-Create and test structured prompting strategies, build RAG pipelines, and implement safety/guardrails.

-Curate and synthesize football-specific data and knowledge graphs for fine-tuning, evaluation, and few-shot performance.

-Deploy dashboards and alerts to monitor drift, toxicity, bias, or degradation in real-world usage.

-Collaborate with engineers, designers, and product teams to bring LLM-powered features to life in production.

Minimum Qualifications

-3+ years experience in applied AI/ML roles with at least 1–2 years focused on LLMs or foundation models.

-Deep understanding of transformer-based models, their limitations, and evaluation strategies.

-Proficiency with Python and key libraries: Hugging Face Transformers, LangChain, OpenAI API, PyTorch/TensorFlow.

-Experience with tracing/debugging LLM outputs using tools like LangSmith, W&B Traces, or custom logs.

-Experience with model distillation, quantization, or fine-tuning on domain-specific tasks.

-Strong communication skills with the ability to translate complex model behavior into actionable insights.

Preferred Qualifications

-Prior experience building or maintaining RAG pipelines (e.g., vector databases, retrieval logic, hybrid search).

-Knowledge of sports analytics or passion for football

-Familiarity with prompt evaluation benchmarks (e.g., TruthfulQA, MMLU, ARC) or custom evaluation harnesses.

-Experience with safety/guardrail frameworks (e.g., OpenAI Moderation, Guardrails.ai, Rebuff).