Staff Research Engineer – RRI Harness

Openkrill; United States; Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Unlimited PTO

Hybrid/Remote configurations

Collaborative culture

Job summary

Sequen is seeking a Staff Research Engineer to own the harness and contribute to applied research for the Recursive Ranking Intelligence (RRI) platform. You will run multi-day experiments on GPUs inside client clouds or Sequen-hosted environments, and build the Python runtime, tools, and adapters that light up long-running agent sessions.

You will design clean contracts with the Go control plane, partner with applied scientists, and deliver reliable, observable systems with a bias toward

Qualifications

• 7+ years building production software with Python and distributed systems.
• Experience with LLMs in agentic loops and scaling challenges.
• Comfort debugging across process, container, and network boundaries.
• Ability to own a system end to end with strong reliability focus.

Responsibilities

• Own the harness: Build and evolve the Python multi-agent runtime across runners, contracts, and adapters.
• Contribute to applied research: Design experiments and ship changes that improve results.
• Build eval infrastructure: Deliver fast eval sets and scorecards for every change.
• Engineer for resilience: Ensure multi-day sessions survive failures and upgrades.
• Deliver observability: Create event streams and cost accounting across sessions.
• Cut waste: Identify idle loops and token waste and fix them in code.
• Design clean contracts: Collaborate on interfaces with the Go control plane and GPU proxy.
• Support clients: Work with scientists and engineers to diagnose production issues.

Skills

Python

Distributed systems

LLM agenting

Kubernetes

Observability

Troubleshooting

Tools

Go

PyTorch

Docker

CI/CD

Job description

STAFF RESEARCH ENGINEER — RRI HARNESS & APPLIED RESEARCH

ABOUT US

Building the ranking intelligence layer for the internet.

Sequen builds Recursive Ranking Intelligence (RRI): an autonomous research engine in which a team of AI agents does the work of an ML research team, autonomously or together with ML researchers, in our clients' own clouds or on Sequen-hosted instances. The models RRI produces serve production traffic for the world's largest retailers, marketplaces and travel platforms, alongside Sequen's ranking platform, which runs frontier ranking models in production at sub-25ms latency and enterprise scale. Each gain compounds into revenue and margin lift measured in hundreds of millions of dollars per customer.

We are a small, highly technical, early-stage team turning recent advances in AI into production systems that operate under unforgiving real-world constraints.

ABOUT THE ROLE

The harness is the runtime at the heart of RRI. It runs the agent team for hours or days, on GPUs inside our clients' own clouds or on Sequen-hosted instances.

We're looking for a Staff Research Engineer to own the harness and contribute to applied research on how RRI's agents work. This is a research engineering role: you will form hypotheses about agent behaviour, run experiments to test them, and ship what works. You will make RRI reliable, safe and performant across long agent sessions, and extend what it can do. Along the way you will build the eval infrastructure that scores every change to the agents before it ships. You will work side by side with our Recursive Self-Improvement Lead, often on the same projects; your profile adds depth in agent engineering and infrastructure.

KEY RESPONSIBILITIES

• Own the harness: Build and evolve the Python multi-agent runtime — orchestration, tool contracts, context and memory management, compaction, and model adapters across LLM providers.
• Contribute to applied research: Design and run experiments on how the agents work — prompting, tool design, context and memory strategies, and model choice — and ship the changes that measurably improve RRI's results.
• Build eval infrastructure: Deliver a fast, fixed eval set, a scorecard with measured noise bands that runs on every harness change, and component ablations.
• Engineer for resilience: Make multi-day sessions survive GPU failures, node replacement, rolling upgrades, provider errors, and credit limits without a human restart.
• Deliver observability: Build event streams, cross-session analysis, per-agent cost and token accounting, and admin tooling over MCP that let us see inside every session.
• Cut waste: Find where agents idle, loop, or burn tokens, and fix it in code.
• Design clean contracts: Work with the owners of our Go control plane and GPU proxy on the interfaces between components.
• Support clients: Partner with applied scientists and forward-deployed engineers to diagnose and fix issues clients hit in production.

ABOUT YOU

• Proven track record: Bring 7+ years of experience building production software, with strong Python and a history of distributed or long-running systems.
• Agentic LLM experience: Have built with LLMs in agentic loops — tool calling, prompt and context management, streaming, retries — and know how they fail at scale.
• System-wide comfort: Debug confidently across process, container, and network boundaries, on Kubernetes and GPU nodes.
• ML fluency: Read a training script, understand a ranking metric, and tell a real regression from noise.
• Extreme ownership: Take absolute accountability for a system end to end, with a deep care for reliability and observability.

STRONG CANDIDATES MAY ALSO BRING

• Applied research experience: Have run ML or LLM experiments end to end, with baselines and ablations, and turned the results into product changes.
• Evaluation infrastructure: Have built test harnesses, benchmarks, experiment tracking, or CI for ML systems.
• Polyglot capabilities: Working knowledge of Go for contributions to the control plane and proxy.
• Customer-controlled deployments: Experience shipping software into on-prem, BYOC, or air-gapped environments and their security constraints.
• ML platform depth: Experience with PyTorch training at scale, GPU scheduling, or ML platforms.
• Open-source footprint: Contributions to agent frameworks, eval harnesses, or ML tooling.

WHAT WE VALUE

• Rigorous systems thinking: You design for the failure case first, because in a multi-day session every rare failure eventually happens.
• Measure, then change: You would rather ship a smaller change with an eval score than a larger one on intuition.
• Pragmatic speed: You move quickly without accumulating debilitating technical debt.

WHAT WE OFFER

• High-impact influence: A staff-level role that owns the core runtime of Sequen's autonomous research engine.
• Pioneering systems: The chance to build production infrastructure for long-horizon AI agents that train models serving enterprise traffic.
• Complete flexibility: Unlimited paid time off, flexible hybrid/remote configurations, and a highly collaborative, world-class engineering culture.