Senior AI Retrieval Platform Engineer

LPL Financial; Fort Mill (SC); On-site; USD 115,000 - 192,000; Full time

Job summary

LPL Financial seeks a Principal Engineer, AI Retrieval & Knowledge Platform to design and build core AI knowledge infrastructure powering enterprise applications. You will work at the intersection of search, vector retrieval, LLMs, and real-time data systems to enable scalable, high-performance AI platforms used by product teams and agents.

The ideal candidate is a strong backend engineer with deep experience in building scalable systems and modern AI-powered retrieval architectures, with a

Qualifications

• 8+ years building large-scale backend, platform, or distributed systems.
• Hands-on experience building search platforms, retrieval systems, indexing pipelines, knowledge systems, or recommendation platforms.
• Strong knowledge of APIs, microservices, Kubernetes, containers, CI/CD, and distributed data processing; proficiency in Java, Python, or Go.

Responsibilities

• Design and build high-scale retrieval systems combining keyword search, semantic search, and vector-based retrieval.
• Develop RAG infrastructure including indexing, retrieval, ranking, and context assembly.
• Build and optimize search indices, vector stores, and hybrid retrieval systems for relevance, latency, and scale.
• Implement advanced ranking, relevance tuning, and personalization pipelines.
• Build streaming and batch pipelines for ingesting and transforming structured and unstructured data.
• Develop enrichment pipelines (chunking, embeddings, metadata extraction, classification).
• Design systems for real-time indexing, incremental updates, and freshness guarantees.
• Optimize data flow, storage, and compute efficiency at scale.
• Build low-latency APIs that expose retrieval and knowledge services to applications and AI agents.
• Develop reusable SDKs and service abstractions for easy integration into product teams.
• Enable context retrieval, query understanding, and response augmentation for downstream AI systems.
• Establish patterns for multi-tenant, scalable platform services.
• Integrate LLMs with retrieval systems for grounded, context-aware experiences.
• Build systems for context construction, prompt augmentation, and response orchestration.
• Implement evaluation frameworks for relevance, grounding quality, and user experience.
• Support use cases like AI assistants, copilots, search experiences, and automation agents.
• Design for low-latency (<100ms), high-throughput, and horizontal scalability.
• Implement caching, sharding, and distributed query execution strategies.
• Build observability pipelines (metrics, logs, tracing).
• Drive resiliency, fault tolerance, and system reliability at scale.
• Lead design and architecture of large-scale AI platform components.
• Mentor engineers on distributed systems, retrieval architectures, and AI engineering practices.

Skills

Distributed systems

Backend engineering

APIs / microservices

Java / Python / Go

Tools

Kubernetes

Containers (Docker)

CI/CD

Elasticsearch/OpenSearch

Job description

LPL Financial seeks a Principal Engineer, AI Retrieval & Knowledge Platform to design and build core AI knowledge infrastructure powering enterprise applications. You will work at the intersection of search, vector retrieval, LLMs, and real-time data systems to enable scalable, high-performance AI platforms used by product teams and agents.

The ideal candidate is a strong backend engineer with deep experience in building scalable systems and modern AI-powered retrieval architectures, with a