AI Search Infrastructure Architect

Olostep Technologies Inc.; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 260,000; Full time

Job summary

Olostep Technologies Inc. is building the next generation search infrastructure for an agentic-first world in San Francisco. This role owns the full search pipeline—from crawling and indexing to processing and ranking results for fast lookup and deep, multi-step research.

You will design scalable, low-latency systems, optimize for freshness and coverage, and instrument the stack with observability to continuously improve relevance and reliability.

Responsibilities

• Design and scale a distributed web crawler optimized for freshness, coverage, and extracting agent-relevant signals.
• Build and maintain a high-throughput indexing pipeline that can process billions of pages with structured metadata.
• Develop query processing and ranking systems that serve both fast search and deep research use cases.
• Build retrieval pipelines that are natively agentic, with awareness of context, multi-hop reasoning, and iterative query refinement.
• Instrument the search stack with observability tooling and use it to continuously improve relevance and reliability.

Job description

Olostep Technologies Inc. is building the next generation search infrastructure for an agentic-first world in San Francisco. This role owns the full search pipeline—from crawling and indexing to processing and ranking results for fast lookup and deep, multi-step research.

You will design scalable, low-latency systems, optimize for freshness and coverage, and instrument the stack with observability to continuously improve relevance and reliability.