Senior Data Engineer — Retrieval & RAG Platform

Solvd, Inc.; United States; Remote; USD 140,000 - 230,000; Full time

Job summary

Solvd, Inc. is a rapidly growing AI-native consulting and technology services firm delivering enterprise transformation across cloud, data, software engineering, and artificial intelligence.

We are seeking a Senior / Staff Data Engineer to own the Bedrock Knowledge Base configuration, OpenSearch Serverless collection, and the retrieval API, with a focus on entitlement-filtered results and scalable operations.

Qualifications

• 4+ years as a software or ML engineer with production RAG or enterprise search experience.
• Hands-on with Amazon Bedrock Knowledge Bases, Retrieve and RetrieveAndGenerate APIs in production.
• Python development shipping Lambda-based APIs behind API Gateway with auth, versioning, and throttling.
• Infrastructure-as-code with Terraform or AWS CDK and CI for client deployments.

Responsibilities

• Stand up Bedrock Knowledge Base and OpenSearch Serverless vector collection from infrastructure code.
• Implement hybrid retrieval with metadata filters and tuning for quality and latency.
• Build retrieval API on API Gateway and Lambda with versioned paths and per-client rate limits.
• Enforce entitlement at query time by filtering on permission metadata.
• Provide Bedrock retrieve-and-generate reference endpoint for pilot validation.
• Build a quality dashboard tracking relevance, grounding, citations, latency, and costs.
• Support contact center pilot weeks 7–9: capture misses, tune, re-run tests.
• Write runbooks for retrieval, embedding updates, and index rebuilds with cost considerations.

Skills

RAG / enterprise search

Python

API design

Terraform / AWS CDK

Cloud dashboards

Tools

Amazon Bedrock

OpenSearch Serverless

Lambda/API Gateway

Terraform

Job description

Solvd Inc. is a rapidly growing AI-native consulting and technology services firm delivering enterprise transformation across cloud, data, software engineering, and artificial intelligence.

Following the acquisition of Tooploox, a premier AI and product development company, Solvd now offers true end-to-end delivery—from strategic advisory and solution design to custom AI development and enterprise-scale implementation. Our capability centers combine deep technical expertise, proven delivery methodologies, and sector-specific knowledge to address complex business challenges quickly and effectively.

We are looking for a Senior / Staff Data Engineer to build what the Agent Assist provider calls: the Bedrock Knowledge Base configuration, the OpenSearch Serverless collection, the versioned retrieval API, and the quality dashboard. You’ll own the retrieval workstream end to end, paired with the ingestion engineer, with technical direction from the Solution Architect.

This role carries the most technical risk in the engagement — entitlement filtering is a severity-one commitment. The mission: ship a retrieval API that returns the right passages, with citations, only to callers entitled to see them, at a latency and cost the client accepts, and that the client can version and operate after week 10.

What you’ll do

• Stand up the Bedrock Knowledge Base and OpenSearch Serverless vector collection in the client’s account from infrastructure code, with the embedding model, chunking strategy, and metadata fields the architect specified.

• Implement hybrid retrieval — vector plus keyword — with metadata filtering by taxonomy, source, status, and review date; tune top‑k, score thresholds, and reranking against the golden question suite.

• Build the retrieval API on API Gateway and Lambda per the Integration Blueprint — versioned paths, authentication for the Agent Assist provider, per‑client rate limits and usage metrics, structured error responses, and a citation payload with source URL, title, section, and last modified date.

• Enforce entitlement at query time by filtering on permission metadata the ingestion pipeline captures — no restricted passage is ever returned to an unentitled caller.

• Provide the Bedrock retrieve‑and‑generate reference endpoint used for pilot validation in weeks 7–9.

• Build the quality dashboard — relevance, grounding, and citation scores trended per run; permission test results; freshness lag per source; API latency and error rate; OpenSearch capacity.

• Support the contact center pilot in weeks 7–9 — capture retrieval misses, tune, and re‑run the suite before each change ships.

• Write the retrieval and API runbooks including model change, re‑embedding, and index rebuild procedures with cost implications.

• Pair with client platform owners in weeks 9–10 and hand over on the transition readiness checklist.

Required skills & experience

• 4+ years as a software or ML engineer, with at least one RAG or enterprise search system delivered to production and measured with an evaluation suite — not just demos.

• Hands‑on with Amazon Bedrock — Knowledge Bases, Retrieve and RetrieveAndGenerate APIs, embedding models — and Amazon OpenSearch Serverless vector search including hybrid queries and metadata filters.

• Python, with experience shipping Lambda‑based APIs behind API Gateway including auth, versioning, and throttling.

• Infrastructure‑as‑code with Terraform or AWS CDK and CI for deployment into a client‑owned account.

• Practical understanding of chunking, embedding model selection, reranking, and the trade‑offs between relevance, latency, and cost.

• Experience building operational dashboards in CloudWatch or an equivalent tool.

Preferred qualifications

• AWS Certified Generative AI Developer Professional or AWS Certified Machine Learning Engineer Associate.

• Experience with RAG evaluation frameworks and metrics — faithfulness, context precision and recall, citation accuracy.

• Experience integrating with a contact center Agent Assist product as an API provider.

• Familiarity with Bedrock Guardrails and permission‑aware retrieval patterns over Microsoft 365 content.

When you join Solvd, you'll…

• Shape real‑world AI‑driven projects across key industries, working with clients from startup innovation to enterprise transformation.

• Be part of a global team with equal opportunities for collaboration across continents and cultures.

• Thrive in an inclusive environment that prioritizes continuous learning, innovation, and ethical AI standards.

Solvd is an equal opportunity employer.