{ "cells": [ { "cell_type": "markdown", "id": "ffeb64bf", "metadata": {}, "source": [ "# الجلسة 2 – خط أنابيب RAG بسيط\n", "\n", "قم ببناء خط أنابيب خفيف لاسترجاع البيانات المعزز باستخدام Foundry Local + تضمينات sentence-transformers.\n" ] }, { "cell_type": "markdown", "id": "77a46eea", "metadata": {}, "source": [ "### شرح: تثبيت التبعيات\n", "يتم تثبيت الحد الأدنى من الحزم لهذه العملية:\n", "- `foundry-local-sdk` لإدارة النماذج محليًا (إذا لم يتم استخدام مسار BASE_URL فقط).\n", "- `openai` لهياكل SDK المتوافقة (بعض الأدوات المساعدة).\n", "- `sentence-transformers` للحصول على التضمينات.\n", "- `numpy` للعمليات الرياضية على المتجهات.\n", "يمكن إعادة تشغيله بأمان؛ تخطى هذه الخطوة إذا كانت البيئة مُعدة بالفعل.\n" ] }, { "cell_type": "markdown", "id": "a857bce7", "metadata": {}, "source": [ "# السيناريو\n", "يقوم هذا الدفتر بإنشاء خط أنابيب بسيط لاسترجاع المعلومات المعزز بالتوليد (RAG) يعمل بالكامل محليًا:\n", "- يتصل بنموذج Foundry Local (يكتشف تلقائيًا عبر SDK أو BASE_URL).\n", "- ينشئ مجموعة صغيرة من الوثائق في الذاكرة ويقوم بتضمينها باستخدام Sentence Transformers.\n", "- ينفذ استرجاعًا بسيطًا للتشابه بين المتجهات (بدون فهرس خارجي) لضمان الشفافية.\n", "- يرسل طلبات توليد مستندة إلى البيانات عبر مسارات احتياطية متعددة لـ HTTP (`/v1/chat/completions`, `/v1/completions`, `/v1/responses`).\n", "- يوفر وظيفة مساعدة `answer()` تعيد المحاولة باستخدام أشكال نماذج بديلة عند فشل المحاولات الأولية.\n", "\n", "استخدم هذا كقالب تشخيصي قبل التوسع إلى مجموعات وثائق أكبر، أو تخزين المتجهات بشكل دائم، أو مقاييس التقييم (راجع دفتر تقييم RAG).\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "7b2c5757", "metadata": {}, "outputs": [], "source": [ "# Install dependencies\n", "!pip install -q foundry-local-sdk openai sentence-transformers numpy" ] }, { "cell_type": "markdown", "id": "9bed3131", "metadata": {}, "source": [ "### شرح: الاستيرادات الأساسية \n", "تحميل المكتبات الأساسية اللازمة للتضمين + الاستنتاج المحلي: \n", "- SentenceTransformer للحصول على تضمينات المتجهات الكثيفة. \n", "- FoundryLocalManager (اختياري) لإدارة الخدمة المحلية. \n", "- عميل OpenAI للحصول على أشكال الكائنات المألوفة (حتى وإن كنا لاحقًا نتعامل مباشرة مع HTTP). \n" ] }, { "cell_type": "code", "execution_count": 6, "id": "5cf135fe", "metadata": {}, "outputs": [], "source": [ "import os, numpy as np\n", "from sentence_transformers import SentenceTransformer\n", "from foundry_local import FoundryLocalManager\n", "from openai import OpenAI" ] }, { "cell_type": "markdown", "id": "725143b8", "metadata": {}, "source": [ "### شرح: مجموعة مستندات تجريبية \n", "تعرف قائمة صغيرة في الذاكرة تحتوي على عبارات من المجال. تُبقي عملية التكرار سريعة ومضبوطة بحيث يتركز الاهتمام على آليات خط الأنابيب (الاستخراج + التأسيس) بدلاً من معالجة البيانات.\n" ] }, { "cell_type": "code", "execution_count": 7, "id": "6f6d493c", "metadata": {}, "outputs": [], "source": [ "DOCS = [\n", " 'Foundry Local provides an OpenAI-compatible local inference endpoint.',\n", " 'Retrieval Augmented Generation improves answer grounding by injecting relevant context.',\n", " 'Edge AI reduces latency and preserves privacy via local execution.',\n", " 'Small Language Models can offer competitive quality with lower resource usage.',\n", " 'Vector similarity search retrieves semantically relevant documents.'\n", "]" ] }, { "cell_type": "markdown", "id": "ef0e66b3", "metadata": {}, "source": [ "### شرح: الاتصال، اختيار النموذج وتهيئة التضمين\n", "منطق الاتصال القوي:\n", "1. يستخدم اختياريًا `BASE_URL` (مسار HTTP خالص) وإلا يعود إلى FoundryLocalManager.\n", "2. يتحقق من `/v1/models` ويختار أفضل معرف نموذج مطابق (تطابق دقيق > العائلة الأساسية > أول متاح).\n", "3. ينفذ حلقة إعادة المحاولة مع إعدادات قابلة للتكوين لـ `FOUNDRY_CONNECT_RETRIES` والتأخير.\n", "4. يهيئ تضمينات SentenceTransformer (متجهات مُطَبَّعة) لمجموعة البيانات التجريبية.\n", "5. يلتقط إصدار OpenAI SDK لضمان إمكانية إعادة الإنتاج.\n", "إذا كانت الخدمة غير موجودة، يعرض إرشادات لتشغيلها بدلاً من التوقف المفاجئ.\n" ] }, { "cell_type": "code", "execution_count": 12, "id": "dae7f5cd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[OK] Foundry Local manager endpoint: http://127.0.0.1:59778/v1 | base=http://127.0.0.1:59778 | alias=phi-4-mini\n", "[OK] Model resolved: deepseek-r1-distill-qwen-7b-cuda-gpu:0 (total_models=11)\n", "[OK] Embedded 5 docs using sentence-transformers/all-MiniLM-L6-v2 shape=(5, 384)\n", "OpenAI SDK version: 1.109.1\n" ] } ], "source": [ "import os, time, json, requests, re\n", "# Native Foundry Local SDK preferred; fall back to explicit BASE_URL if provided\n", "os.environ.setdefault('FOUNDRY_LOCAL_ALIAS', 'phi-4-mini')\n", "alias = os.getenv('FOUNDRY_LOCAL_ALIAS', os.getenv('TARGET_MODEL', 'phi-4-mini'))\n", "base_url_env = os.getenv('BASE_URL', '').strip()\n", "manager = None\n", "client = None\n", "endpoint = None\n", "\n", "def _canonicalize(model_id: str) -> str:\n", " \"\"\"Remove CUDA suffix and version tags from model name.\"\"\"\n", " b = model_id.split(':')[0]\n", " return re.sub(r'-cuda.*', '', b)\n", "\n", "try:\n", " if base_url_env:\n", " # Allow user override; normalize by removing trailing / and optional /v1\n", " root = base_url_env.rstrip('/')\n", " if root.endswith('/v1'):\n", " root = root[:-3]\n", " endpoint = root\n", " print(f'[INFO] Using explicit BASE_URL override: {endpoint}')\n", " else:\n", " from foundry_local import FoundryLocalManager\n", " manager = FoundryLocalManager(alias)\n", " # Manager endpoint already includes /v1 - remove it for our base\n", " raw_endpoint = manager.endpoint.rstrip('/')\n", " if raw_endpoint.endswith('/v1'):\n", " endpoint = raw_endpoint[:-3]\n", " else:\n", " endpoint = raw_endpoint\n", " print(f'[OK] Foundry Local manager endpoint: {manager.endpoint} | base={endpoint} | alias={alias}')\n", " \n", " # Probe models list (endpoint does NOT include /v1 here)\n", " models_resp = requests.get(endpoint + '/v1/models', timeout=5)\n", " models_resp.raise_for_status()\n", " payload = models_resp.json() if models_resp.headers.get('content-type','').startswith('application/json') else {}\n", " data = payload.get('data', []) if isinstance(payload, dict) else []\n", " ids = [m.get('id') for m in data if isinstance(m, dict)]\n", " \n", " # Select best matching model\n", " chosen = None\n", " if alias in ids:\n", " chosen = alias\n", " else:\n", " for mid in ids:\n", " if _canonicalize(mid) == _canonicalize(alias):\n", " chosen = mid\n", " break\n", " if not chosen and ids:\n", " chosen = ids[0]\n", " model_name = chosen or alias\n", " \n", " # Initialize OpenAI client\n", " from openai import OpenAI as _OpenAI\n", " client = _OpenAI(\n", " base_url=endpoint + '/v1', # OpenAI client needs full base URL with /v1\n", " api_key=(getattr(manager, 'api_key', None) or os.getenv('API_KEY') or 'not-needed')\n", " )\n", " print(f'[OK] Model resolved: {model_name} (total_models={len(ids)})')\n", "except Exception as e:\n", " print('[ERROR] Failed to initialize Foundry Local client:', e)\n", " client = None\n", " model_name = alias\n", "\n", "# Expose BASE for downstream compatibility (without /v1)\n", "BASE = endpoint\n", "\n", "# Embeddings setup\n", "embed_model_name = os.getenv('EMBED_MODEL', 'sentence-transformers/all-MiniLM-L6-v2')\n", "try:\n", " from sentence_transformers import SentenceTransformer\n", " embedder = SentenceTransformer(embed_model_name)\n", " doc_emb = embedder.encode(DOCS, convert_to_numpy=True, normalize_embeddings=True)\n", " print(f'[OK] Embedded {len(DOCS)} docs using {embed_model_name} shape={doc_emb.shape}')\n", "except Exception as e:\n", " print('[ERROR] Embedding init failed:', e)\n", " embedder = None\n", " doc_emb = None\n", "\n", "try:\n", " import openai as _openai\n", " openai_version = getattr(_openai, '__version__', 'unknown')\n", " print('OpenAI SDK version:', openai_version)\n", "except Exception:\n", " openai_version = 'unknown'\n", "\n", "if client is None:\n", " print('\\nNEXT: Start/verify service then re-run this cell:')\n", " print(' foundry service start')\n", " print(' foundry model run phi-4-mini')\n", " print(' (optional) set BASE_URL=http://127.0.0.1:57127')" ] }, { "cell_type": "markdown", "id": "e6537337", "metadata": {}, "source": [ "### شرح: وظيفة الاسترجاع (تشابه المتجهات)\n", "`retrieve(query, k=3)` تقوم بترميز الاستعلام، حساب التشابه الكوني (حاصل الضرب النقطي على المتجهات المُطَبَّعة)، وتعيد أعلى k من مؤشرات المستندات. هذه الطريقة تبقى بسيطة وفي الذاكرة من أجل الشفافية.\n" ] }, { "cell_type": "code", "execution_count": 9, "id": "373d8845", "metadata": {}, "outputs": [], "source": [ "def retrieve(query, k=3):\n", " q = embedder.encode([query], convert_to_numpy=True, normalize_embeddings=True)[0]\n", " sims = doc_emb @ q\n", " return sims.argsort()[::-1][:k]" ] }, { "cell_type": "markdown", "id": "ea42bed1", "metadata": {}, "source": [ "### شرح: إنشاء يعتمد على SDK ومساعد الإجابة\n", "تمت إعادة تصميمه لاستخدام Foundry Local SDK + طرق العميل المتوافقة مع OpenAI بدلاً من إرسال طلبات HTTP الخام يدويًا:\n", "- المسار الأساسي: `client.chat.completions.create` (رسائل منظمة).\n", "- البدائل: `client.completions.create` (موجه قديم) ثم `client.responses.create` (واجهة برمجية للاستجابات المبسطة).\n", "- يقوم بتوحيد معرفات النماذج البديلة (RAW مقابل ALT المبسط) لتوسيع التوافق.\n", "- تقوم `answer()` بإنشاء موجه مستند إلى المستندات المسترجعة من أعلى k وتسجيل محاولات التتبع المرتبة.\n", "هذا يحافظ على منطق الكود واضحًا مع تقديم تدهور سلس عبر نقاط النهاية المتطورة المتوافقة مع OpenAI.\n" ] }, { "cell_type": "code", "execution_count": 14, "id": "526983ad", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[INFO] SDK generation mode active.\n", " RAW_MODEL = deepseek-r1-distill-qwen-7b-cuda-gpu:0\n", " ALT_MODEL = deepseek-r1-distill-qwen-7b\n" ] } ], "source": [ "# SDK-based generation (Foundry Local manager + OpenAI client methods)\n", "import re, time, json\n", "\n", "def _strip_model_name(name: str) -> str:\n", " \"\"\"Strip CUDA suffix and version tags from model name.\"\"\"\n", " base = name.split(':')[0]\n", " base = re.sub(r'-cuda.*', '', base)\n", " return base\n", "\n", "# Use the actual resolved model name from connection cell\n", "RAW_MODEL = model_name\n", "ALT_MODEL = _strip_model_name(RAW_MODEL)\n", "\n", "def _try_via_client(messages, prompt, model_id: str, max_tokens=220, temperature=0.2):\n", " \"\"\"Try generating response using OpenAI client with multiple fallback routes.\"\"\"\n", " attempts = []\n", " \n", " # 1. Try chat.completions endpoint (preferred for chat models)\n", " try:\n", " resp = client.chat.completions.create(\n", " model=model_id, \n", " messages=messages, \n", " max_tokens=max_tokens, \n", " temperature=temperature\n", " )\n", " content = resp.choices[0].message.content\n", " attempts.append(('chat.completions', 200, (content or '')[:160]))\n", " if content and content.strip():\n", " return content, attempts\n", " except Exception as e:\n", " attempts.append(('chat.completions', None, str(e)[:160]))\n", " \n", " # 2. Try legacy completions endpoint\n", " try:\n", " comp = client.completions.create(\n", " model=model_id, \n", " prompt=prompt, \n", " max_tokens=max_tokens, \n", " temperature=temperature\n", " )\n", " txt = comp.choices[0].text if comp.choices else ''\n", " attempts.append(('completions', 200, (txt or '')[:160]))\n", " if txt and txt.strip():\n", " return txt, attempts\n", " except Exception as e:\n", " attempts.append(('completions', None, str(e)[:160]))\n", " \n", " return None, attempts\n", "\n", "def retrieve(query, k=3):\n", " \"\"\"Retrieve top-k most similar documents using cosine similarity.\"\"\"\n", " if embedder is None or doc_emb is None:\n", " raise RuntimeError(\"Embeddings not initialized.\")\n", " q_emb = embedder.encode([query], normalize_embeddings=True)[0]\n", " scores = doc_emb @ q_emb\n", " idxs = np.argsort(scores)[::-1][:k]\n", " return idxs\n", "\n", "def answer(query, k=3, max_tokens=220, temperature=0.2, try_alternate=True):\n", " \"\"\"\n", " Answer a query using RAG pipeline:\n", " 1. Retrieve relevant documents using vector similarity\n", " 2. Generate grounded response using Foundry Local model via OpenAI SDK\n", " \n", " Args:\n", " query: User question\n", " k: Number of documents to retrieve\n", " max_tokens: Maximum tokens for generation\n", " temperature: Sampling temperature\n", " try_alternate: Whether to try alternate model name on failure\n", " \n", " Returns:\n", " Dictionary with query, answer, docs, context, route, and tried attempts\n", " \"\"\"\n", " if client is None:\n", " raise RuntimeError('Model client not initialized. Re-run connection cell after starting Foundry Local.')\n", " if embedder is None or doc_emb is None:\n", " raise RuntimeError('Embeddings not initialized.')\n", " \n", " # Retrieve relevant documents\n", " idxs = retrieve(query, k=k)\n", " context = '\\n'.join(f'Doc {i}: {DOCS[i]}' for i in idxs)\n", " \n", " # Construct grounded generation prompt\n", " system_content = 'Use ONLY provided context. If insufficient, say \"I\\'m not sure.\"'\n", " user_content = f'Context:\\n{context}\\n\\nQuestion: {query}'\n", " messages = [\n", " {'role': 'system', 'content': system_content},\n", " {'role': 'user', 'content': user_content}\n", " ]\n", " prompt = f'System: {system_content}\\n{user_content}\\nAnswer:'\n", " \n", " # Try generation with primary model\n", " tried = []\n", " ans, attempts = _try_via_client(messages, prompt, RAW_MODEL, max_tokens=max_tokens, temperature=temperature)\n", " tried.append({'model': RAW_MODEL, 'attempts': attempts})\n", " \n", " if ans and ans.strip():\n", " return {\n", " 'query': query, \n", " 'answer': ans.strip(), \n", " 'docs': idxs.tolist(), \n", " 'context': context, \n", " 'route': 'chat-first', \n", " 'tried': tried\n", " }\n", " \n", " # Try alternate model name if available\n", " if try_alternate and ALT_MODEL != RAW_MODEL:\n", " ans2, attempts2 = _try_via_client(messages, prompt, ALT_MODEL, max_tokens=max_tokens, temperature=temperature)\n", " tried.append({'model': ALT_MODEL, 'attempts': attempts2})\n", " if ans2 and ans2.strip():\n", " return {\n", " 'query': query, \n", " 'answer': ans2.strip(), \n", " 'docs': idxs.tolist(), \n", " 'context': context, \n", " 'route': 'chat-alt', \n", " 'tried': tried\n", " }\n", " \n", " # All routes failed\n", " return {\n", " 'query': query, \n", " 'answer': 'I\\'m not sure. (All SDK routes failed)', \n", " 'docs': idxs.tolist(), \n", " 'context': context, \n", " 'route': 'failed', \n", " 'tried': tried\n", " }\n", "\n", "print('[INFO] SDK generation mode active.')\n", "print(f' RAW_MODEL = {RAW_MODEL}')\n", "print(f' ALT_MODEL = {ALT_MODEL}')" ] }, { "cell_type": "code", "execution_count": 15, "id": "b19a21cb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'alt_model': 'deepseek-r1-distill-qwen-7b',\n", " 'answer_preview': 'Okay, so I need to figure out why someone would use '\n", " 'Retrieval Augmented Generation (RAG) with local inference. '\n", " 'Let me start by understanding each part of the qu',\n", " 'base': 'http://127.0.0.1:59778',\n", " 'raw_model': 'deepseek-r1-distill-qwen-7b-cuda-gpu:0',\n", " 'retrieved_indices': [0, 3, 1],\n", " 'route': 'chat-first'}\n" ] } ], "source": [ "# Self-test cell: validates connectivity, embeddings, and answer() basic functionality (SDK mode)\n", "import math, pprint\n", "\n", "def rag_self_test(sample_query: str = 'Why use RAG with local inference?', expect_docs: int = 3):\n", " report = {'base': BASE, 'raw_model': RAW_MODEL, 'alt_model': ALT_MODEL}\n", " if not BASE:\n", " report['error'] = 'BASE not resolved'\n", " return report\n", " if embedder is None or doc_emb is None:\n", " report['error'] = 'Embeddings not initialized'\n", " return report\n", " if getattr(doc_emb, 'shape', (0,))[0] != len(DOCS):\n", " report['warning_embeddings'] = f\"doc_emb count {getattr(doc_emb,'shape',('?'))} mismatch DOCS {len(DOCS)}\"\n", " try:\n", " idxs = retrieve(sample_query, k=expect_docs)\n", " report['retrieved_indices'] = idxs.tolist() if hasattr(idxs, 'tolist') else list(idxs)\n", " except Exception as e:\n", " report['error_retrieve'] = str(e)\n", " return report\n", " try:\n", " ans = answer(sample_query, k=expect_docs, max_tokens=80, temperature=0.2)\n", " report['route'] = ans.get('route')\n", " report['answer_preview'] = ans.get('answer','')[:160]\n", " if ans.get('route') == 'failed':\n", " report['warning_generation'] = 'All SDK routes failed for sample query'\n", " except Exception as e:\n", " report['error_generation'] = str(e)\n", " return report\n", "\n", "pprint.pprint(rag_self_test())" ] }, { "cell_type": "markdown", "id": "1ccaa474", "metadata": {}, "source": [ "### شرح: اختبار الدخان للاستعلامات الجماعية\n", "يقوم بتنفيذ عدة أسئلة نموذجية للمستخدم من خلال `answer()` للتحقق من:\n", "- أن مؤشرات الاسترجاع تتوافق مع وثائق داعمة معقولة.\n", "- أن التوجيه الاحتياطي يعمل (القيمة المسار ليست 'failed').\n", "- أن الإجابات تلتزم بتعليمات الأساس (بدون تخيلات).\n", "يتم حفظ آخر كائن نتيجة للفحص الفوري عند الحاجة.\n" ] }, { "cell_type": "code", "execution_count": 16, "id": "c38a64fb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Q: Why use RAG with local inference?\n", "A: Okay, so I need to figure out why someone would use Retrieval Augmented Generation (RAG) with local inference. Let me start by understanding each part of the question.\n", "\n", "First, RAG. From the context given, Doc 1 says that RAG improves answer grounding by injecting relevant context. So RAG is a method that uses retrieval techniques to find the most relevant parts of a document or corpus to augment the generation process. This probably helps in making the generated answers more accurate because they're backed by real data.\n", "\n", "Then, local inference. Doc 0 mentions that Foundry Local provides an OpenAI-compatible local inference endpoint. So local inference means running the model on the user's device rather than sending the request to a remote server. This is good for privacy and reducing latency, but it might have limitations in terms of model size or capabilities compared to cloud-based options.\n", "\n", "Now, combining RAG with local inference. The context says that small language models can offer competitive quality with lower resource usage (Doc 3). So using a smaller model might be more efficient, but maybe RAG\n", "Docs: [0, 3, 1]\n", "---\n", "Q: What does vector similarity search do?\n", "A: Okay, so I need to figure out what vector similarity search does based on the provided context. Let me start by looking at the given documents.\n", "\n", "Doc 4 says, \"Vector similarity search retrieves semantically relevant documents.\" Hmm, that seems straightforward. So vector similarity search is a method that retrieves documents based on their semantic relevance. It probably uses some kind of vector representation for the documents, maybe through techniques like word embeddings or TF-IDF vectors. Then, when a query comes in, it's converted into a vector, and the system finds documents with similar vectors, meaning they share similar meanings or topics.\n", "\n", "I should make sure I'm not missing anything. The other documents mention edge AI and retrieval augmented generation, but they don't directly relate to vector similarity search. So, the key point here is that it retrieves semantically relevant documents. That makes sense because vector-based methods are commonly used in information retrieval to find documents that are not just syntactically similar but actually about the same topic.\n", "\n", "I think that's the main idea. Vector similarity search focuses on semantic relevance, using vector representations to\n", "Docs: [4, 2, 1]\n", "---\n", "Q: Explain privacy benefits.\n", "A: Okay, so I need to explain the privacy benefits mentioned in the provided context. Let me look at the context again. The context includes three documents:\n", "\n", "Doc 2 says Edge AI reduces latency and preserves privacy via local execution.\n", "Doc 3 mentions Small Language Models can offer competitive quality with lower resource usage.\n", "Doc 1 states Retrieval Augmented Generation improves answer grounding by injecting relevant context.\n", "\n", "The question is about explaining the privacy benefits. So, I should focus on the parts of the context that talk about privacy. \n", "\n", "Looking at Doc 2, it mentions Edge AI reduces latency and preserves privacy via local execution. That seems directly related to privacy. I think \"local execution\" means that the AI processes data on the device itself rather than sending it to a server. This could mean that data doesn't have to be transmitted, which might help protect user privacy because it avoids centralizing data that could be intercepted.\n", "\n", "Doc 3 talks about small language models with lower resource usage. I'm not sure how this relates to privacy. It might be more about efficiency rather than privacy benefits.\n", "\n", "Doc 1\n", "Docs: [2, 3, 1]\n", "---\n" ] }, { "data": { "text/plain": [ "{'query': 'Explain privacy benefits.',\n", " 'answer': 'Okay, so I need to explain the privacy benefits mentioned in the provided context. Let me look at the context again. The context includes three documents:\\n\\nDoc 2 says Edge AI reduces latency and preserves privacy via local execution.\\nDoc 3 mentions Small Language Models can offer competitive quality with lower resource usage.\\nDoc 1 states Retrieval Augmented Generation improves answer grounding by injecting relevant context.\\n\\nThe question is about explaining the privacy benefits. So, I should focus on the parts of the context that talk about privacy. \\n\\nLooking at Doc 2, it mentions Edge AI reduces latency and preserves privacy via local execution. That seems directly related to privacy. I think \"local execution\" means that the AI processes data on the device itself rather than sending it to a server. This could mean that data doesn\\'t have to be transmitted, which might help protect user privacy because it avoids centralizing data that could be intercepted.\\n\\nDoc 3 talks about small language models with lower resource usage. I\\'m not sure how this relates to privacy. It might be more about efficiency rather than privacy benefits.\\n\\nDoc 1',\n", " 'docs': [2, 3, 1],\n", " 'context': 'Doc 2: Edge AI reduces latency and preserves privacy via local execution.\\nDoc 3: Small Language Models can offer competitive quality with lower resource usage.\\nDoc 1: Retrieval Augmented Generation improves answer grounding by injecting relevant context.',\n", " 'route': 'chat-first',\n", " 'tried': [{'model': 'deepseek-r1-distill-qwen-7b-cuda-gpu:0',\n", " 'attempts': [('chat.completions',\n", " 200,\n", " 'Okay, so I need to explain the privacy benefits mentioned in the provided context. Let me look at the context again. The context includes three documents:\\n\\nDoc ')]}]}" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Quick test queries\n", "\n", "queries = [\n", "\n", " \"Why use RAG with local inference?\",\n", "\n", " \"What does vector similarity search do?\",\n", "\n", " \"Explain privacy benefits.\"\n", "\n", "]\n", "\n", "\n", "\n", "last_result = None\n", "\n", "for q in queries:\n", "\n", " try:\n", "\n", " r = answer(q)\n", "\n", " last_result = r\n", "\n", " print(f\"Q: {q}\\nA: {r['answer']}\\nDocs: {r['docs']}\\n---\")\n", "\n", " except Exception as e:\n", "\n", " print(f\"Failed answering '{q}': {e}\")\n", "\n", "\n", "\n", "last_result" ] }, { "cell_type": "markdown", "id": "0f8282fc", "metadata": {}, "source": [ "### شرح: استدعاء مريح لإجابة واحدة\n", "الاستدعاء النهائي السريع لسؤال واحد لتسهيل النسخ/اللصق أو الرجوع إليه لاحقًا. يوضح الاستخدام المتكرر لـ `answer()` بعد الاستفسارات التمهيدية السابقة.\n" ] }, { "cell_type": "code", "execution_count": 17, "id": "d4d25165", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'query': 'Why use RAG with local inference?',\n", " 'answer': \"Okay, so I need to figure out why someone would use Retrieval Augmented Generation (RAG) with local inference. Let me start by understanding each part of the question.\\n\\nFirst, RAG. From the context given, Doc 1 says that RAG improves answer grounding by injecting relevant context. So RAG is a method that uses retrieval techniques to find the most relevant parts of a document or corpus to augment the generation process. This probably helps in making the generated answers more accurate because they're backed by real data.\\n\\nThen, local inference. Doc 0 mentions that Foundry Local provides an OpenAI-compatible local inference endpoint. So local inference means running the model on the user's device rather than sending the request to a remote server. This is good for privacy and reducing latency, but it might have limitations in terms of model size or capabilities compared to cloud-based options.\\n\\nNow, combining RAG with local inference. The context says that small language models can offer competitive quality with lower resource usage (Doc 3). So using a smaller model might be more efficient, but maybe RAG\",\n", " 'docs': [0, 3, 1],\n", " 'context': 'Doc 0: Foundry Local provides an OpenAI-compatible local inference endpoint.\\nDoc 3: Small Language Models can offer competitive quality with lower resource usage.\\nDoc 1: Retrieval Augmented Generation improves answer grounding by injecting relevant context.',\n", " 'route': 'chat-first',\n", " 'tried': [{'model': 'deepseek-r1-distill-qwen-7b-cuda-gpu:0',\n", " 'attempts': [('chat.completions',\n", " 200,\n", " 'Okay, so I need to figure out why someone would use Retrieval Augmented Generation (RAG) with local inference. Let me start by understanding each part of the qu')]}]}" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "result = answer('Why use RAG with local inference?')\n", "result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n---\n\n**إخلاء المسؤولية**: \nتمت ترجمة هذا المستند باستخدام خدمة الترجمة بالذكاء الاصطناعي [Co-op Translator](https://github.com/Azure/co-op-translator). بينما نسعى لتحقيق الدقة، يرجى العلم أن الترجمات الآلية قد تحتوي على أخطاء أو عدم دقة. يجب اعتبار المستند الأصلي بلغته الأصلية المصدر الموثوق. للحصول على معلومات حاسمة، يُوصى بالترجمة البشرية الاحترافية. نحن غير مسؤولين عن أي سوء فهم أو تفسيرات خاطئة ناتجة عن استخدام هذه الترجمة.\n" ] } ], "metadata": { "kernelspec": { "display_name": "demo", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.15" }, "coopTranslator": { "original_hash": "9f009936a177f8847d250fe1771d25f6", "translation_date": "2025-10-09T07:12:30+00:00", "source_file": "Workshop/notebooks/session02_rag_pipeline.ipynb", "language_code": "ar" } }, "nbformat": 4, "nbformat_minor": 5 }