Sr. Applied AI Engineer

1 O.C. Tanner Company; Salt Lake City (UT); On-site; USD 140,000 - 180,000; Full time

Job summary

O.C. Tanner is seeking an Applied AI Engineer who can turn AI capability into secure, governed production systems at scale.

This role sits at the intersection of software engineering, product experience, AI platform engineering, and responsible AI, partnering with Product, UX, Design, Architecture, Security, and Engineering to build AI experiences that are useful, understandable, reliable, and safe to operate in production.

Qualifications

• 5+ years of software engineering experience with Python proficiency.
• 2+ years building production ML or agentic AI systems.
• 1+ year of hands-on experience with agentic frameworks (LangGraph, CrewAI, AutoGen, or equivalent).
• Built production AI systems including agents, MCP servers, multi-step reasoning, and multi-turn conversation.
• Deployed RAG systems including embeddings, vector databases, hybrid search, and retrieval optimization.
• Designed LLM strategies covering tool calling, structured outputs, prompt engineering, and context window management.
• Implemented AI safety and evaluation pipelines including bias detection, PII leakage, faithfulness scoring, toxicity, and prompt injection mitigation.

Responsibilities

• Design, build, deploy, and support production-grade agentic AI systems that operate against explicit goals, constraints, policies, and guardrails.
• Build agent orchestration patterns for multi-step workflows, tool calling, MCP servers, state management, memory, retries, recovery paths, and human-in-the-loop controls.
• Partner with Product, UX, Design, Architecture, Security, and Engineering to create AI experiences aligned with user workflows.
• Design user-centered AI interactions, including conversational flows, feedback loops, confidence handling, explainability, and boundaries for autonomous behavior.
• Develop and operate RAG systems grounding model behavior in enterprise knowledge (ingestion, chunking, embeddings, vector/hybrid retrieval, reranking, citations).
• Define and implement evaluation frameworks: offline tests, regression suites, adversarial testing, and production monitoring.
• Instrument agentic systems for observability: traces of model calls, prompts, tool usage, decisions, latency, cost, errors, and feedback.
• Establish safeguards for responsible AI use: prompt injection defense, data access controls, PII protection, bias detection, and audit logging.
• Mentor engineers on applied AI practices and stay current with emerging AI platforms and standards.

Skills

Python

Production ML/Agentic AI

LangGraph / LangChain

Agentic frameworks

LLM prompts engineering

AI safety & evaluation

Model optimization

Education

Bachelor's degree in CS/ML or related field

Tools

LangChain

LangGraph

AutoGen

AWS

Terraform

OpenTelemetry

Kubernetes

Job description

O.C. Tanner is the global leader in software and services that improve workplace culture through meaningful employee experiences. Our Culture Cloud is a suite of apps designed to enhance the employee experience with strategic recognition, service awards, wellbeing, leadership, and events that help people thrive at work. Our Culture by Design approach provides expert services to organizations looking to create great workplaces. Our global team of 1,500 people hail from 58 countries and speak 62 languages. As programmers, researchers, designers, client professionals and craftspeople we create the tech, tools and awards that connect employees to purpose at thousands of companies.

Join us as we help people all over the world thrive at work.

About the Role

AI is becoming part of the product and platform architecture that we need to build, operate, and scale. We are looking for an Applied AI Engineer who can turn AI capability into secure, measurable, governed production systems, not prototypes or demos. This person will help define how O.C. Tanner builds agentic systems that pursue goals, use tools, follow guardrails, recover from failure, and deliver real value inside user workflows. This role sits at the intersection of software engineering, product experience, AI platform engineering, and responsible AI.

You will partner with Product, UX, Design, Architecture, Security, and Engineering to build AI experiences that are useful, understandable, reliable, and safe to operate in production.

Responsibilities

• Design, build, deploy, and support production-grade agentic AI systems that operate against explicit goals, constraints, policies, and guardrails.
• Build agent orchestration patterns for multi-step workflows, tool calling, MCP servers, state management, memory, retries, recovery paths, and human-in-the-loop controls.
• Partner closely with Product, UX, Design, Architecture, Security, and Engineering teams to create AI experiences that are useful, understandable, reliable, and aligned with real user workflows.
• Design user-centered AI interactions, including conversational flows, feedback loops, confidence handling, explainability, graceful failure modes, escalation paths, and clear boundaries for autonomous behavior.
• Develop and operate RAG systems that ground model behavior in enterprise knowledge, including ingestion, chunking, embeddings, vector and hybrid retrieval, reranking, retrieval evaluation, and citation or traceability strategies.
• Define and implement evaluation frameworks for AI systems, including offline test sets, regression suites, adversarial testing, groundedness and faithfulness scoring, task completion metrics, and production quality monitoring.
• Instrument agentic systems for observability, including traces of model calls, prompts, tool usage, decisions, retrieved context, latency, cost, errors, and user feedback.
• Establish safeguards for responsible AI use, including prompt injection defense, data access controls, PII protection, bias and toxicity detection, misuse prevention, audit logging, and policy enforcement.
• Optimize model selection, prompts, context windows, caching, routing, inference patterns, latency, throughput, reliability, and cost across production workloads.
• Mentor engineers on applied AI practices, including prompt and context engineering, agent design, RAG, evaluation, safety, observability, and production support.
• Stay current with emerging AI platforms, frameworks, models, and standards.

Qualifications

Required qualifications:

• 5+ years of software engineering experience with strong Python proficiency.
• 2+ years building production ML or agentic AI systems.
• 1+ year of hands-on experience with agentic frameworks (LangGraph, CrewAI, AutoGen, or equivalent).
• Built production AI systems including agents, MCP servers, multi-step reasoning, and multi-turn conversation.
• Deployed RAG systems including embedding models, vector databases, hybrid search, and retrieval optimization.
• Designed LLM strategies covering tool calling, structured outputs, prompt engineering, and context window management.
• Implemented AI safety and evaluation pipelines covering bias detection, PII leakage, faithfulness scoring, toxicity, and prompt injection mitigation.
• Optimized models for inference efficiency, latency, and cost management.
• Strongly preferred: Bachelor's degree in Computer Science, Machine Learning, or a related field.
• Strongly preferred: AWS Certified Machine Learning Engineer – Associate or equivalent.
• Strongly preferred: Cloud AI infrastructure management using AWS services and Terraform.
• Strongly preferred: AI observability experience with OpenTelemetry, Langfuse, or equivalent.

Technical Stack

Python / FastAPI microservices; LangChain / LangGraph; GraphQL / REST; PostgreSQL / Redis; Kafka; Kubernetes; AWS Bedrock; OpenTelemetry; Terraform.

We create inspiring workplaces for some of the biggest and best companies in the world. And we do it within our own teams every day. That’s one reason we made the Fortune 100 Best Companies to Work For list in 2021. Join us and watch people thrive at work—including you. With seven global offices and employees working around the world, we’re committed to creating an atmosphere where every person can share their talents and reach their potential.