Senior AI Agent Runtime & Evals Engineer - Remote Equity

Sytrex; San Francisco (CA); On-site; USD 140,000 - 230,000; Full time

Benefits offered by this job

Remote work

In-person hackathons

Offsites

Job summary

Braven is agentic infrastructure for reinsurance. A Senior engineer on the Platform team you own the agent runtime and the evaluation system around it. Paid in USD. Remote, with in person hackathons and offsites.

You will own the agent runtime and evaluation system, building resilience across models and ensuring reliable production deployments. Strong engineering background in a TypeScript stack is preferred, with English as the working language.

Qualifications

• You have run agents in production, not demos: tool calling, context management, streaming, failure recovery.
• You have evaluated agentic systems: designed eval sets, graders, metrics and CI gates, using tooling like LangSmith.
• You can debug non determinism: probabilistic, provider specific, intermittent failures do not scare you.
• Strong general engineering. We are a TypeScript shop (Bun, Fastify), but strong engineers from other stacks pick it up fast.
• Strong written and spoken English, our working language.
• Nice to have: LangGraph / LangChain or similar agent frameworks, low level experience with a major model provider, and an eye for inference cost and security.

Responsibilities

• Own the agent runtime: tool calling, context management, streaming, and graceful recovery when a run fails.
• Build resilience across multiple model providers (e.g. Bedrock, Azure OpenAI): fallback by error type, retries, cancellation, and cost control.
• Own the evaluation system: offline datasets, automated graders, CI regression gates, and online evaluation over real production traces, tracking task success, cost and latency.
• Keep the agent predictable in production by hardening non deterministic failures: malformed tool schemas, context overflow, provider specific edge cases.
• Treat evals as a ship gate, not decoration. Every change ships with its eval and a green regression gate.
• Make context management token accurate (offload, summarization, budget caps) instead of fragile heuristics.

Skills

Agent runtime

Context management

Streaming

Recovery

Tools

LangSmith

LangChain

Bedrock

Azure OpenAI

Job description

Braven is agentic infrastructure for reinsurance. A Senior engineer on the Platform team you own the agent runtime and the evaluation system around it. Paid in USD. Remote, with in person hackathons and offsites.

You will own the agent runtime and evaluation system, building resilience across models and ensuring reliable production deployments. Strong engineering background in a TypeScript stack is preferred, with English as the working language.