Freelance RAG and prompt engineer (m/f/d)

vona KI- und Softwareagentur; United States; Remote; USD 83,000 - 165,000; Full time

Benefits offered by this job

Fully remote work

Project-based engagement

Flexible hours

Potential follow-on projects

Job summary

vona KI- und Softwareagentur seeks a freelance RAG engineer to work on client projects remotely from Europe. You’ll design retrieval pipelines, prepare data, and build prompts that ground responses to sources.

You’ll evaluate answer quality, compare cloud versus local hosting, and collaborate with clients to explain trade-offs. Strong Python and LangChain/LlamaIndex skills are required, with fluency in German or English.

Qualifications

• Several years of software development experience, preferably including Python.
• Confident use of LangChain or LlamaIndex and at least one vector database or vector search solution.
• A systematic approach to measuring answer quality.
• Ability to explain technical trade-offs to clients without a technical background.
• Confident communication in German or English.
• Freelance or self-employed status and a base in Germany, Austria, or Switzerland.

Responsibilities

• Design retrieval pipelines by connecting data sources, preparing documents, choosing chunking strategies, and selecting embeddings and retrieval methods.
• Implement solutions with an appropriate framework, such as LangChain, LangGraph, or LlamaIndex.
• Create and version prompts that keep responses grounded in sources and make gaps in available information clear.
• Build representative question sets and evaluate retrieval and answer quality throughout development.
• Assess whether a cloud-hosted or locally run model best fits each project.

Skills

Software development

RAG systems

German or English communication

Tools

LangChain

LlamaIndex

Vector databases

Job description

Role overview

Build retrieval-augmented generation systems that help AI assistants answer questions using product catalogs and knowledge bases. You’ll work on client projects, shaping how information is prepared and retrieved, how prompts guide responses, and how answer quality is measured before launch and after changes.

Responsibilities

• Design retrieval pipelines by connecting data sources, preparing documents, choosing chunking strategies, and selecting embeddings and retrieval methods.
• Implement solutions with an appropriate framework, such as LangChain, LangGraph, or LlamaIndex.
• Create and version prompts that keep responses grounded in sources and make gaps in available information clear.
• Build representative question sets and evaluate retrieval and answer quality throughout development.
• Assess whether a cloud-hosted or locally run model best fits each project.

Requirements

• Several years of software development experience, preferably including Python, and at least one RAG system deployed in production.
• Confident use of LangChain or LlamaIndex and at least one vector database or vector search solution.
• A systematic approach to measuring answer quality.
• Ability to explain technical trade-offs to clients without a technical background.
• Confident communication in German or English.
• Freelance or self-employed status and a base in Germany, Austria, or Switzerland.

Nice to have

• Experience with LangGraph, agent frameworks, Hugging Face, or operating local models.

Benefits and work setup

• Freelance, project-based engagement, fully remote from Germany, Austria, or Switzerland; scope and hours are agreed for each project. Work is expected to start from 11November2026, with possible extension.
• Client projects have defined goals, with a dedicated contact and regular check-ins.
• Relevant AI tools are provided, and you work from your own setup with flexible hours tied to the workload.
• Follow‑on projects may be available if the collaboration goes well, though they are not guaranteed.