--- title: RAG knowledge base tags: [rag, tools, agents] date: 2025-01-20 authors: [ao-anam] --- # RAG knowledge base When users ask your avatar questions, the underlying LLM generates answers from its training data. But what if users ask about your specific product, your company policies, or documentation that the LLM has never seen? This is where RAG comes in. RAG (Retrieval-Augmented Generation) lets your avatar search your documents and use that information to answer questions accurately. ## What is RAG? RAG works in two steps: 1. **Retrieval** - When a user asks a question, the system searches your documents to find relevant passages 2. **Generation** - The LLM uses those passages as context when generating its response Without RAG, if you asked "What's the return policy?", the LLM would either make something up or say it doesn't know. With RAG, it searches your policy documents, finds the relevant section, and gives an accurate answer. The "augmented" part is important. The LLM still generates natural, conversational responses. It just has access to your documents as a reference. ## How Anam's knowledge tools work Anam handles the RAG pipeline for you. Here's what happens when a user asks a question: 1. You upload documents to a knowledge folder 2. Anam processes and indexes them for semantic search 3. You create a knowledge tool that points to that folder 4. You attach the tool to your persona 5. When the LLM decides it needs information, it calls the tool 6. Anam searches your documents and returns relevant chunks 7. The LLM uses those chunks to generate an accurate response The LLM decides when to use the tool based on your tool description. If a user asks a casual question like "how are you?", it won't search. If they ask "what are the product dimensions?", it will. ## Setting up a knowledge base ### Creating a knowledge folder Knowledge folders organize your documents. You might have one folder for product docs, another for FAQs, and another for policies. ```typescript const response = await fetch("https://api.anam.ai/v1/knowledge/groups", { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.ANAM_API_KEY}`, }, body: JSON.stringify({ name: "Product Documentation", description: "Technical guides and product specifications", }), }); const { id: folderId } = await response.json(); ``` The description helps you remember what's in the folder. It doesn't affect search results. ### Uploading documents Anam supports PDF, TXT, MD, DOCX, CSV, JSON, and LOG files up to 50MB. The upload process has three steps for security and reliability. First, request a signed upload URL: ```typescript const uploadResponse = await fetch( `https://api.anam.ai/v1/knowledge/groups/${folderId}/documents/presigned-upload`, { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.ANAM_API_KEY}`, }, body: JSON.stringify({ filename: "product-manual.pdf", contentType: "application/pdf", fileSize: file.size, }), } ); const { uploadUrl, documentId } = await uploadResponse.json(); ``` Then upload the file directly to the signed URL: ```typescript await fetch(uploadUrl, { method: "PUT", headers: { "Content-Type": file.type }, body: file, }); ``` Finally, confirm the upload so Anam can start processing: ```typescript await fetch( `https://api.anam.ai/v1/knowledge/documents/${documentId}/confirm-upload`, { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.ANAM_API_KEY}`, }, body: JSON.stringify({ fileSize: file.size }), } ); ``` Processing takes about 30 seconds. The document status changes from `PROCESSING` to `READY` when it's searchable. ### How documents get processed Anam automatically chunks your documents for search. Different file types are handled differently: - **PDF, TXT, MD, DOCX** - Split into paragraphs. Each paragraph becomes a searchable chunk. - **CSV** - Each row becomes a chunk. Good for product catalogs or FAQs in spreadsheet format. - **JSON** - Kept intact. Useful for structured data. - **LOG** - Each line becomes a chunk. You don't need to pre-process your documents. But the structure of your documents affects search quality. More on that later. ## Creating a knowledge tool A knowledge tool tells the LLM how and when to search your documents. You define it like any other tool, but with `type: "server"` and `subtype: "knowledge"`. ```typescript const personaConfig = { name: "Support Agent", avatarId: "your-avatar-id", voiceId: "your-voice-id", systemPrompt: `You are a helpful product support agent. When users ask questions about our products, use the search_docs tool to find accurate information. If you can't find relevant information, say so honestly.`, tools: [ { type: "server", subtype: "knowledge", name: "search_docs", description: "Search product documentation when users ask questions about features, specifications, installation, troubleshooting, or usage.", documentFolderIds: [folderId], }, ], }; ``` The `documentFolderIds` array specifies which folders to search. You can include multiple folders if your knowledge is spread across them. The tool description is important. It tells the LLM when to use the tool. Be specific about the types of questions that should trigger a search. ### Using the tool in a session Include the persona config when creating a session token: ```typescript const response = await fetch("https://api.anam.ai/v1/auth/session-token", { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.ANAM_API_KEY}`, }, body: JSON.stringify({ personaConfig }), }); const { sessionToken } = await response.json(); ``` The knowledge tool works automatically. You don't need to handle any events in your client code. When the LLM calls the tool, Anam performs the search and returns results to the LLM, which then generates a response. ## Writing good documents for RAG Search quality depends on how you structure your documents. Here are the principles that matter: ### Make paragraphs self-contained Each paragraph might be retrieved without its surrounding context. If a paragraph references "the steps above" or "as mentioned earlier", the LLM won't have that context. ```text Bad - relies on context: Follow the steps above to install. Then configure as follows... Good - self-contained: After installing the application, open Settings > Configuration to set up your preferences. ``` ### Use clear headings Headings help Anam understand document structure. They also give the LLM context about what section information came from. ```text # Product Installation Guide ## System Requirements The application requires Windows 10 or later, macOS 12 or later, or Ubuntu 20.04 or later. Minimum 4GB RAM and 500MB disk space. ## Installation Steps Download the installer from our website. Run the installer and follow the prompts. The default installation location is recommended for most users. ``` ### Keep files focused A single 200-page PDF covering everything is harder to search than 10 focused documents. Consider splitting large documents by topic: - `installation-guide.pdf` - `troubleshooting-guide.pdf` - `api-reference.pdf` - `faq.pdf` ### Use descriptive filenames Filenames are metadata that can help with search relevance: - `password-reset-guide.pdf` (good) - `document-final-v2.pdf` (not helpful) ## System prompts for RAG The system prompt guides how the LLM uses retrieved information. Here's a pattern that works well: ```typescript const systemPrompt = `You are a product support specialist for Acme Corp. When users ask product questions: 1. Use the search_docs tool to find accurate information 2. Answer based on what you find in the documentation 3. If the documentation doesn't cover the question, say "I don't have that information in our documentation" and offer to connect them with support Never make up product specifications, prices, or policies. If you're not sure, say so. Keep responses conversational and concise since they'll be spoken aloud.`; ``` The key instruction is telling the LLM to admit when it doesn't have information. Without this, LLMs tend to generate plausible-sounding but incorrect answers. ## Testing your knowledge base Before going live, test with questions that should retrieve from your documents: ```typescript const testQuestions = [ // Should find specific information "What are the system requirements?", "How do I reset my password?", "What's included in the pro plan?", // Should acknowledge missing information "What's the weather like today?", "Compare your product to CompetitorX", ]; ``` For each question, verify that: 1. The avatar searches when it should 2. The response matches your documentation 3. It admits when information isn't available You can test search directly using the debug modal in Anam Lab (`Ctrl+Shift+K` on the knowledge page) to see what chunks are returned for a query. ## Organizing multiple folders For larger knowledge bases, organize folders by domain: ``` Product Documentation/ - installation-guide.pdf - feature-overview.pdf - api-reference.pdf Support FAQs/ - billing-faq.pdf - troubleshooting-faq.pdf Policies/ - privacy-policy.pdf - terms-of-service.pdf ``` You can create focused tools that search specific folders: ```typescript tools: [ { type: "server", subtype: "knowledge", name: "search_product_docs", description: "Search for technical product information, features, and how-to guides", documentFolderIds: [productDocsFolderId], }, { type: "server", subtype: "knowledge", name: "search_policies", description: "Search company policies including privacy policy, terms of service, and refund policy", documentFolderIds: [policiesFolderId], }, ] ``` This helps the LLM choose the right tool for each question. Technical questions go to product docs, policy questions go to policies. ## Common issues **Search returns no results** - Check that documents are in `READY` status (processing takes ~30 seconds) - Verify the folder ID in your tool matches where you uploaded documents - Try the debug modal to test queries directly **LLM doesn't use the tool** - Check your tool description. It should clearly describe when to use the tool. - Update your system prompt to instruct the LLM to use the tool for relevant questions. **Responses are inaccurate** - The retrieved chunks might not contain the answer. Check what's being retrieved using the debug modal. - Your documents might need better structure. See the "Writing good documents" section. - Add explicit instructions in your system prompt to only answer from documentation.