Staff Software Engineer (Applied Artificial Intelligence)

Okta; San Francisco (CA); Remote; USD 180,000 - 230,000; Full time

Benefits offered by this job

Work from home opportunities

Health + Wellness

Financial Benefits

Pay + Incentives

Time Off

Everyday Living

Resources

Job summary

Okta Inc. in San Francisco is seeking an engineering leader to help build Digital Workers, a suite of autonomous agents for identity operations and security.

You will own harnesses and evaluations, integrate OpenAI and Anthropic APIs, design tests, measure accuracy and cost, and ship reliable tools customers can trust.

This role blends startup ownership with the resources of a public company, offering remote work options and the chance to scale AI for Okta’s 20,000+ customers.

Qualifications

• Relates to building AI agents and evaluation systems (no specific qualification text provided).

Responsibilities

• Build harnesses for agent planning, tool use, memory, and recovery during long-running work.
• Integrate model APIs (OpenAI, Anthropic) and select models based on quality, latency, and cost.
• Design and run evaluation datasets and tests for valid findings, access decisions, and onboarding.
• Measure false positives, incomplete actions, and production failures to drive regression testing.
• Ensure secure, policy-checked writes and evidenced states before completion.

Job description

• The team that built Jeeva AI is building Digital Workers, a new suite of autonomous agents for identity operations and security inside Okta
• The team works directly with Todd McKinnon, Okta’s co-founder and CEO, with the opportunity to bring this product to Okta’s 20,000+ customers
• You will help shape the product with the ownership and pace of a startup and the resources of a public company
• You will build the harnesses and evaluations that make these workers reliable
• They investigate risk, use tools, wait for approvals and complete tasks across enterprise systems
• Bring depth in agent execution or model quality and help turn it into working software customers can trust
• Build the agent harness for planning, tool use, memory, context compaction, interruption and recovery during long-running work
• Integrate OpenAI and Anthropic model APIs. Select models using measured quality, latency and cost, with clear failure handling
• Evaluate existing harnesses and build only the missing pieces. Keep the harness connected to AWS workflow execution and approval state
• Keep authorization and sensitive writes behind application policy checks. Require evidence of the resulting state before marking work complete
• Build evaluation datasets and replay tests for valid findings, access decisions, app setup and AI agent onboarding
• Measure false positives, missed risks, unsafe actions, completion rate and cost. Turn production failures into regression tests
• Test prompt injection, stale evidence, conflicting instructions and unsafe tool use. Protect tenant boundaries in retrieval and memory
• Improve prompts, retrieval, tools and model routing. Evaluate fine-tuning or other post-training methods when results justify them and data use is approved
• Staff: Lead a domain such as harnesses, evaluations or model quality across workers and implement its critical components

Benefits

• Work from home opportunities
• Health + Wellness
• Financial Benefits
• Pay + Incentives
• Time Off
• Everyday Living
• Resources

Production builder: You have shipped AI software and know what changes when real users depend on itStrong owner: You follow an improvement through code, evaluation, release and customer useCareful evaluator: You investigate failures and use evidence to judge qualityHands-on experience building agent workflows or evaluation systems for tool-using LLM applications. For harness work, relevant experience includes Claude Agent SDK, OpenAI Codex SDK, Strands Agents, LangChain Deep Agents, LangGraph or a custom implementationExperience integrating model APIs, structured outputs and asynchronous services. Experience with OpenAI or Anthropic is especially relevantStrong Python engineering skills and experience shipping an LLM application that uses toolsDepth in agent execution or evaluation systems, with working code and failure analysis you can explainOur core stack is Python FastAPI, AWS Step Functions, Amazon Bedrock AgentCore, PostgreSQL on Amazon RDS and pgvector. OpenAI and Anthropic model APIs are part of the integration planWe plan to add an agent harness and a dedicated analytics layer using ClickHouse or BigQuery. The harness choice is open. Experience with equivalent tools also countsAbility to distinguish a plausible model response from a supported finding or verified actionOpen-source contributions, agent observability, security evaluations or synthetic evaluation dataFine-tuning, preference data, post-training evaluation or model distillation using approved datasetsIdentity and security workflows. Foundation-model research experience is optional