Remote Staff AI Engineer: Agent Harness & Model Quality

Okta Inc.; San Francisco (CA); Remote; USD 180,000 - 230,000; Full time

Benefits offered by this job

Work from home opportunities

Health + Wellness

Financial Benefits

Pay + Incentives

Time Off

Everyday Living

Resources

Job summary

Okta Inc. in San Francisco is seeking an engineering leader to help build Digital Workers, a suite of autonomous agents for identity operations and security.

You will own harnesses and evaluations, integrate OpenAI and Anthropic APIs, design tests, measure accuracy and cost, and ship reliable tools customers can trust.

This role blends startup ownership with the resources of a public company, offering remote work options and the chance to scale AI for Okta’s 20,000+ customers.

Qualifications

• Relates to building AI agents and evaluation systems (no specific qualification text provided).

Responsibilities

• Build harnesses for agent planning, tool use, memory, and recovery during long-running work.
• Integrate model APIs (OpenAI, Anthropic) and select models based on quality, latency, and cost.
• Design and run evaluation datasets and tests for valid findings, access decisions, and onboarding.
• Measure false positives, incomplete actions, and production failures to drive regression testing.
• Ensure secure, policy-checked writes and evidenced states before completion.

Job description

Okta Inc. in San Francisco is seeking an engineering leader to help build Digital Workers, a suite of autonomous agents for identity operations and security.

You will own harnesses and evaluations, integrate OpenAI and Anthropic APIs, design tests, measure accuracy and cost, and ship reliable tools customers can trust.

This role blends startup ownership with the resources of a public company, offering remote work options and the chance to scale AI for Okta’s 20,000+ customers.