Staff Software Engineer - AI Search Infrastructure

Hidden Jobs; United States; Remote; USD 160,000 - 230,000; Full time

Benefits offered by this job

Competitive compensation

International, collaborative team

Job summary

Hidden Jobs is seeking a senior or staff-level software engineer to own the indexing and data processing layer of a purpose-built search engine for AI agents. The role focuses on offline and nearline pipelines capable of moving tens of gigabytes per second, ensuring data freshness and efficient queryability.

You will design and operate core indexing systems, optimize storage formats and update mechanisms, and collaborate with ML teams to meet retrieval and ranking requirements in a

Qualifications

• Five+ years of experience building production backend or data infrastructure systems.
• Strong proficiency in Go; working knowledge of C++ or Rust is a plus.
• Experience with large-scale data processing at multi-GB/s throughput and petabyte-scale datasets.

Responsibilities

• Design, build, and operate large-scale indexing systems and data pipelines at the core of the search infrastructure.
• Develop and tune indexing strategies balancing performance, freshness, and resource efficiency.

Skills

Go

C++

Rust

Distributed systems

Data pipelines

Observability

Tools

Spark

Flink

MapReduce

Beam

Databases

Job description

Hidden Jobs is seeking a senior or staff-level software engineer to own the indexing and data processing layer of a purpose-built search engine for AI agents. The role focuses on offline and nearline pipelines capable of moving tens of gigabytes per second, ensuring data freshness and efficient queryability.

You will design and operate core indexing systems, optimize storage formats and update mechanisms, and collaborate with ML teams to meet retrieval and ranking requirements in a