Staff Machine Learning Engineer, Generative AI (Auth0)

Okta; Toronto; On-site; CAD 140,000 - 180,000; Full time

Job summary

Okta is seeking a Staff Machine Learning Engineer to shape and accelerate Generative AI strategy across model development, infrastructure, and platform services. You will architect production-ready ML/GenAI systems at scale, ranging from LLM-powered features to reusable components for cross-team use.

You will collaborate with Product, Security, and Platform Engineering to deliver AI-powered experiences that are both innovative and trustworthy, while building scalable infrastructure for

Qualifications

• 7+ years of software development experience with Python
• Hands-on ML experience from feature engineering to fine-tuning models
• Experience with Generative AI platforms (AWS Bedrock, OpenAI, Anthropic)
• Deep understanding of retrieval-augmented generation (RAG), embeddings, and knowledge-base workflows
• Hands-on experience with AI agent frameworks (LiteLLM, LangGraph, LangChain, LlamaIndex, MCP)
• Familiarity with ML frameworks (FastAPI, PyTorch, TensorFlow, Spark ML) and workflow tools (Airflow)
• Experience defining evaluation metrics, pipelines, and feedback loops for ML/GenAI systems
• Proven ability to collaborate with product and engineering teams to drive greenfield initiatives

Responsibilities

• Architect, design, and deploy robust ML/GenAI systems with integration to platform services
• Develop scalable LLMOps pipelines in production
• Balance simplicity, flexibility, reliability, and performance in technical decisions
• Lead initiatives to tune, optimize, and deploy agentic applications with security focus
• Partner with Product, Security, and Platform Engineering to design AI-powered experiences
• Design and implement scalable infrastructure for large-scale GenAI use cases
• Collaborate with product managers, researchers, and engineers to deliver secure and high-quality AI/ML systems

Skills

Python

Go/Typescript

Machine learning

GenAI platforms

RAG

LangChain

Tools

LightLLM

LangGraph

LangChain

LlamaIndex

MCP

PyTorch

FastAPI

Job description

Secure Every Identity, from AI to Human

Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence.

This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk.

The Team

Have you ever considered what powers the intelligent features behind seamless product experiences? The GenAI team is at the forefront of enabling AI-powered security and intelligent innovation across our organization. From crafting AI powered security services, to intuitive generative AI-powered chat experiences that provide instant product support, and developing the best developer experience around authentication for generative AI and AI agents, our team is instrumental in bringing the transformative power of AI to life. We collaborate closely with the Machine Learning team and various product teams to ensure the seamless and secure delivery of AI-enhanced features that provide real value to our users.

The Opportunity

As a Staff Machine Learning Engineer on the Generative AI team, you will help shape, architect, and accelerate our Generative AI strategy by contributing across the stack of model development, infrastructure, and platform services. You'll drive design and implementation of production-ready AI/ML systems at scale: ranging from LLM-powered features to reusable components that other teams across Okta can build on.

You will have the opportunity to

Architect, design, and deploy robust Machine Learning & GenAI systems, ensuring seamless integration with diverse platform services and establishing scalable LLMOps pipelines in production.

Drive technical decision making while striving to hit the right balance between factors such as simplicity, flexibility, reliability, and performance.

Lead initiatives to tune, optimize, and deploy agentic applications in production with a focus on performance, reliability, and security.

Partner with Product, Security, and Platform Engineering teams to design AI-powered experiences that are both innovative and trustworthy.

Design and implement scalable infrastructure and platform services for large-scale Generative AI use cases.

Collaborate cross-functionally with product managers, researchers, and engineers to deliver secure, high-quality, and scalable AI/ML systems.

What you will do

Spearhead the design of scalable, observable ML and Generative AI systems that integrate retrieval, inference, and evaluation pipelines.

Develop and iterate on structured prompting, context retrieval, and RAG workflows that improve accuracy, safety, and cost efficiency in Claude-based systems.

Build and refine automated evaluation pipelines to measure model quality, correctness, groundedness, and safety in production.

Implement schema validation, structured output enforcement, and other guardrails that keep AI outputs reliable, auditable, and compliant with enterprise standards.

Mentor and coach engineers, contributing to the growth of the team and the larger engineering community.

What you bring

• 7+ years of software development experience, with strong programming expertise in Python (and familiarity with Go or Typescript a plus).
• Hands-on experience with applied machine learning, from feature engineering to training and fine-tuning models.
• Hands-on experience with modern Generative AI platforms (AWS Bedrock, OpenAI, Anthropic, etc.).
• Deep understanding of retrieval-augmented generation (RAG), embeddings, and knowledge-base workflows.
• Hands-on experience with LiteLLM, LangGraph, LangChain, LlamaIndex, MCP, or other related AI agent frameworks.
• Familiarity with ML frameworks (FastAPI, PyTorch, TensorFlow, Spark ML) and workflow orchestration tools (Airflow, etc.).
• Experience defining evaluation metrics, pipelines, and feedback loops for ML/GenAI systems.
• Proven ability to collaborate with product and engineering teams to drive greenfield initiatives forward, navigate unknowns, and iterate quickly and frequently.
• Experience building tools or infrastructure for AI/ML applications, with a deep understanding of the developer lifecycle in an AI-native world.

Nice to haves

• Experience integrating AI-driven systems with identity, authentication, or security products.
• Exposure to ethical AI, model risk, or compliance frameworks.
• Familiarity with evaluation datasets, synthetic data generation, or LLM-as-a-judge methods.