Principal Engineer, AI Retrieval – Knowledge Platforms

Jobtailor; New York (NY); On-site; USD 150,000 - 225,000; Full time

Job summary

Jobtailor is seeking a Senior Backend Architect in New York to design and build large-scale retrieval systems and RAG infrastructure. The role focuses on real-time indexing, vector stores, and hybrid retrieval, with close collaboration across AI, data, and product teams to deliver grounded, context-aware experiences.

You will mentor engineers, drive CI/CD and infrastructure-as-code adoption, and ensure low latency, high throughput platform capabilities in regulated enterprise environments.

Qualifications

• 8+ years building large-scale backend, platform, or distributed systems.
• Hands-on experience building search platforms, retrieval systems, indexing pipelines, knowledge systems, or recommendation platforms.
• Strong knowledge of APIs, microservices, Kubernetes, containers, CI/CD, and distributed data processing.
• Experience building enterprise platforms in highly regulated environments such as Wealth Management, Brokerage / Broker-Dealer, Banking, Capital Markets, Insurance, FinTech, or Healthcare.
• Knowledge of RAG (Retrieval Augmented Generation), semantic search, vector databases, embeddings, LLM integration patterns, knowledge graphs, real-time data platforms, Kafka / streaming systems, observability and monitoring, platform APIs, event-driven architecture, and multi-tenant platform design

Responsibilities

• Design and build high-scale retrieval systems and backend platforms.
• Lead architecture of large-scale AI platform components.
• Integrate LLMs with retrieval systems for grounded, context-aware experiences.
• Develop observability pipelines for metrics, logs, tracing, and performance.
• Mentor engineers and drive CI/CD, infra-as-code, and automated testing.
• Partner with AI, data, and product teams to shape platform capabilities.
• Build indexing pipelines, vector stores, enrichment pipelines, and real-time indexing.
• Create low-latency, highly available knowledge-service APIs.

Skills

Large-Scale Backend

Search Platform Engineering

APIs & Microservices

Java, Python, or Go

RAG & Semantic Search

Tools

Kubernetes

CI/CD

Containers

Kafka

Event-Driven Architecture

Distributed Data Processing

Job description

• Design and build high-scale retrieval systems combining keyword, semantic, and vector-based retrieval
• Develop RAG infrastructure including indexing, retrieval, ranking, and context assembly
• Build and optimize search indices, vector stores, and hybrid retrieval systems
• Implement ranking, relevance tuning, and personalization pipelines
• Build streaming and batch pipelines for structured and unstructured data
• Develop enrichment pipelines for chunking, embeddings, metadata extraction, and classification
• Design real-time indexing, incremental updates, and freshness guarantees
• Build low-latency, highly available retrieval and knowledge-service APIs
• Develop reusable SDKs and service abstractions for product teams
• Integrate LLMs with retrieval systems for grounded, context-aware experiences
• Build context construction, prompt augmentation, response orchestration, and evaluation frameworks
• Design for low latency, high throughput, horizontal scalability, resiliency, and fault tolerance
• Build observability pipelines for metrics, logs, tracing, performance, and usage insights
• Lead architecture of large-scale AI platform components
• Mentor engineers and drive adoption of CI/CD, infrastructure-as-code, and automated testing
• Partner with AI, data, and product teams to shape intelligent platform capabilities

Requirements

• Minimum of 8 years building large-scale backend, platform, or distributed systems
• Hands-on experience building search platforms, retrieval systems, indexing pipelines, knowledge systems, or recommendation platforms
• Strong knowledge of APIs, microservices, Kubernetes, containers, CI/CD, and distributed data processing
• Programming experience in Java, Python, or Go
• Experience building enterprise platforms in highly regulated environments such as Wealth Management, Brokerage / Broker-Dealer, Banking, Capital Markets, Insurance, FinTech, or Healthcare
• Knowledge of RAG (Retrieval Augmented Generation), semantic search, vector databases, embeddings, LLM integration patterns, knowledge graphs, real-time data platforms, Kafka / streaming systems, observability and monitoring, platform APIs, event-driven architecture, and multi-tenant platform design

Core Competencies

Demonstrates expertise in building large-scale retrieval systems and backend platforms, with a strong focus on indexing, ranking, and real-time data processing. Proficient in integrating LLMs and developing observability pipelines to enhance system performance and user experience.

Highest-signal resume keywords

• Large-Scale Backend Development
• Search Platform Engineering
• APIs and Microservices
• Java, Python, or Go Programming
• RAG and Semantic Search Knowledge

Hard Skills

• Indexing Pipelines
• Retrieval Systems
• Knowledge Systems
• Ranking and Relevance Tuning
• Hybrid Retrieval Systems
  • Indexing Pipelines
  • Retrieval Systems
  • Knowledge Systems
  • Ranking and Relevance Tuning
  • Hybrid Retrieval Systems
  • Data Processing
  • Observability Pipelines
  • Real-Time Indexing
  • Low-Latency API Development
  • Embedding and Metadata Extraction

  Soft Skills

  • Mentoring Engineers
  • Collaboration with Product Teams

  Industry Keywords

  • Wealth Management
  • Banking
  • FinTech
  • Healthcare
  • Capital Markets
  • Brokerage
  • Broker-Dealer
  • Regulated Environments

  Tools & Technologies

  • Kubernetes
  • CI/CD
  • Containers
  • Kafka
  • Event-Driven Architecture
  • Distributed Data Processing