{ "cells": [ { "cell_type": "markdown", "id": "af028554", "metadata": {}, "source": [ "# نظام توجيه يعتمد على النوايا باستخدام Foundry Local SDK\n", "\n", "**نظام توجيه متعدد النماذج محسّن لوحدة المعالجة المركزية**\n", "\n", "يستعرض هذا الدفتر نظام توجيه ذكي يختار تلقائيًا أفضل نموذج لغة صغير بناءً على نية المستخدم. مثالي لسيناريوهات النشر على الحافة حيث ترغب في استخدام نماذج متخصصة متعددة بكفاءة.\n", "\n", "## 🎯 ما ستتعلمه\n", "\n", "- **اكتشاف النوايا**: تصنيف المطالبات تلقائيًا (كود، تلخيص، تصنيف، عام)\n", "- **اختيار النموذج الذكي**: التوجيه إلى النموذج الأكثر قدرة لكل مهمة\n", "- **تحسين وحدة المعالجة المركزية**: نماذج فعالة من حيث الذاكرة تعمل على أي جهاز\n", "- **إدارة النماذج المتعددة**: الاحتفاظ بالنماذج محملة باستخدام `--retain true`\n", "- **أنماط الإنتاج**: منطق إعادة المحاولة، معالجة الأخطاء، وتتبع الرموز\n", "\n", "## 📋 نظرة عامة على السيناريو\n", "\n", "يستعرض هذا النمط:\n", "\n", "1. **اكتشاف النوايا**: تصنيف كل مطالبة مستخدم (كود، تلخيص، تصنيف، أو عام)\n", "2. **اختيار النموذج**: اختيار النموذج الصغير الأكثر ملاءمة تلقائيًا بناءً على القدرات\n", "3. **التنفيذ المحلي**: التوجيه إلى النماذج التي تعمل محليًا عبر خدمة Foundry Local\n", "4. **واجهة موحدة**: نقطة دخول واحدة للدردشة توجه إلى نماذج متخصصة متعددة\n", "\n", "**مثالي لـ**: عمليات النشر على الحافة مع نماذج متخصصة متعددة حيث ترغب في توجيه الطلبات بذكاء دون اختيار النموذج يدويًا.\n", "\n", "## 🔧 المتطلبات الأساسية\n", "\n", "- **Foundry Local** مثبتة والخدمة تعمل\n", "- **Python 3.8+** مع pip\n", "- **8GB+ RAM** (يوصى بـ 16GB+ للنماذج المتعددة)\n", "- **وحدة workshop_utils** (في ../samples/)\n", "\n", "## 🚀 البدء السريع\n", "\n", "سيقوم الدفتر بـ:\n", "1. اكتشاف ذاكرة النظام لديك\n", "2. التوصية بنماذج وحدة المعالجة المركزية المناسبة\n", "3. تحميل النماذج تلقائيًا باستخدام `--retain true`\n", "4. التحقق من جاهزية جميع النماذج\n", "5. توجيه المطالبات التجريبية إلى النماذج المتخصصة\n", "\n", "**الوقت المتوقع للإعداد**: 5-7 دقائق (يشمل تحميل النماذج)\n" ] }, { "cell_type": "markdown", "id": "b4aa55f0", "metadata": {}, "source": [ "## 📦 الخطوة 1: تثبيت التبعيات\n", "\n", "قم بتثبيت حزمة Foundry Local SDK الرسمية والمكتبات المطلوبة:\n", "\n", "- **foundry-local-sdk**: حزمة Python الرسمية لإدارة النماذج محليًا\n", "- **openai**: واجهة برمجة تطبيقات متوافقة مع OpenAI لإكمال المحادثات\n", "- **psutil**: لاكتشاف ومراقبة ذاكرة النظام\n" ] }, { "cell_type": "code", "execution_count": 107, "id": "2929c9f5", "metadata": {}, "outputs": [], "source": [ "# Install core dependencies\n", "!pip install -q foundry-local-sdk openai psutil" ] }, { "cell_type": "markdown", "id": "990799e7", "metadata": {}, "source": [ "## 💻 الخطوة 2: اكتشاف ذاكرة النظام\n", "\n", "قم باكتشاف الذاكرة المتوفرة في النظام لتحديد نماذج وحدة المعالجة المركزية (CPU) التي يمكن تشغيلها بكفاءة. يضمن ذلك اختيار النموذج الأمثل لجهازك.\n" ] }, { "cell_type": "code", "execution_count": 108, "id": "ff58f1ee", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🖥️ System Memory Information\n", "======================================================================\n", "Total Memory: 63.30 GB\n", "Available Memory: 16.19 GB\n", "\n", "✅ High Memory System (32GB+)\n", " Can run 3-4 models simultaneously\n", "\n", "📋 Recommended Model Aliases for Your System:\n", " • phi-4-mini\n", " • phi-3.5-mini\n", " • qwen2.5-0.5b\n", " • qwen2.5-coder-0.5b\n", "\n", "💡 About Model Aliases:\n", " ✓ Use base alias (e.g., phi-4-mini, not phi-4-mini-cpu)\n", " ✓ Foundry Local automatically selects CPU variant for your hardware\n", " ✓ No GPU required - optimized for CPU inference\n", " ✓ Predictable memory usage and consistent performance\n", "======================================================================\n" ] } ], "source": [ "import psutil\n", "\n", "# Get system memory information\n", "total_memory_gb = psutil.virtual_memory().total / (1024**3)\n", "available_memory_gb = psutil.virtual_memory().available / (1024**3)\n", "\n", "print('🖥️ System Memory Information')\n", "print('=' * 70)\n", "print(f'Total Memory: {total_memory_gb:.2f} GB')\n", "print(f'Available Memory: {available_memory_gb:.2f} GB')\n", "print()\n", "\n", "# Recommend models based on available memory\n", "# Using model aliases - Foundry Local will automatically select CPU variant\n", "model_aliases = []\n", "\n", "if total_memory_gb >= 32:\n", " model_aliases = ['phi-4-mini', 'phi-3.5-mini', 'qwen2.5-0.5b', 'qwen2.5-coder-0.5b']\n", " print('✅ High Memory System (32GB+)')\n", " print(' Can run 3-4 models simultaneously')\n", "elif total_memory_gb >= 16:\n", " model_aliases = ['phi-4-mini', 'qwen2.5-0.5b', 'phi-3.5-mini']\n", " print('✅ Medium Memory System (16-32GB)')\n", " print(' Can run 2-3 models simultaneously')\n", "elif total_memory_gb >= 8:\n", " model_aliases = ['qwen2.5-0.5b', 'phi-3.5-mini']\n", " print('⚠️ Lower Memory System (8-16GB)')\n", " print(' Recommended: 2 smaller models')\n", "else:\n", " model_aliases = ['qwen2.5-0.5b']\n", " print('⚠️ Limited Memory System (<8GB)')\n", " print(' Recommended: Use only smallest model')\n", "\n", "print()\n", "print('📋 Recommended Model Aliases for Your System:')\n", "for model in model_aliases:\n", " print(f' • {model}')\n", "\n", "print()\n", "print('💡 About Model Aliases:')\n", "print(' ✓ Use base alias (e.g., phi-4-mini, not phi-4-mini-cpu)')\n", "print(' ✓ Foundry Local automatically selects CPU variant for your hardware')\n", "print(' ✓ No GPU required - optimized for CPU inference')\n", "print(' ✓ Predictable memory usage and consistent performance')\n", "print('=' * 70)" ] }, { "cell_type": "markdown", "id": "69590b94", "metadata": {}, "source": [ "## 🤖 الخطوة 3: تحميل النماذج تلقائيًا\n", "\n", "يقوم هذا الجزء تلقائيًا بـ:\n", "1. تشغيل خدمة Foundry Local (إذا لم تكن قيد التشغيل)\n", "2. تحميل النماذج الموصى بها باستخدام `--retain true` (يحتفظ بعدة نماذج في الذاكرة)\n", "3. التحقق من جاهزية جميع النماذج باستخدام SDK\n", "\n", "⏱️ **الوقت المتوقع**: 3-5 دقائق لجميع النماذج\n" ] }, { "cell_type": "code", "execution_count": null, "id": "543fd976", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🚀 Automatic Model Loading with SDK Verification\n", "======================================================================\n", "📋 Loading 3 models: ['phi-4-mini', 'phi-3.5-mini', 'qwen2.5-0.5b']\n", "💡 Using model aliases - Foundry will load CPU variants automatically\n", "\n", "📡 Step 1: Checking Foundry Local service...\n", " ✅ Service is already running\n", "\n", "🤖 Step 2: Loading models with retention...\n", " [1/3] Starting phi-4-mini...\n", " ✅ phi-4-mini loading in background\n", " [2/3] Starting phi-3.5-mini...\n", " ✅ phi-3.5-mini loading in background\n", " [3/3] Starting qwen2.5-0.5b...\n", " ✅ qwen2.5-0.5b loading in background\n", "\n", "✅ Step 3: Verifying models (this may take 2-3 minutes)...\n", "======================================================================\n", "\n", " Attempt 1/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 2/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 3/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 4/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 5/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 6/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 7/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 8/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 9/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 10/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 11/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 12/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 13/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 14/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 15/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 16/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 17/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 18/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 19/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 20/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 21/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 22/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 23/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 24/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 25/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 26/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 27/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 28/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 29/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", " Attempt 30/30...\n", " ⚠️ phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n", " ⚠️ qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n", "\n", "======================================================================\n", "📦 Final Status: 0/3 models ready\n", " ❌ phi-4-mini - NOT READY\n", " ❌ phi-3.5-mini - NOT READY\n", " ❌ qwen2.5-0.5b - NOT READY\n", "\n", "⚠️ Some models not ready. Check: foundry model ls\n" ] } ], "source": [ "import subprocess\n", "import time\n", "import sys\n", "import os\n", "\n", "# Add samples directory for workshop_utils (Foundry SDK pattern)\n", "sys.path.append(os.path.join('..', 'samples'))\n", "\n", "print('🚀 Automatic Model Loading with SDK Verification')\n", "print('=' * 70)\n", "\n", "# Use top 3 recommended models (aliases)\n", "# Foundry will automatically load CPU variants\n", "REQUIRED_MODELS = model_aliases[:3]\n", "print(f'📋 Loading {len(REQUIRED_MODELS)} models: {REQUIRED_MODELS}')\n", "print('💡 Using model aliases - Foundry will load CPU variants automatically')\n", "print()\n", "\n", "# Step 1: Ensure Foundry Local service is running\n", "print('📡 Step 1: Checking Foundry Local service...')\n", "try:\n", " result = subprocess.run(['foundry', 'service', 'status'], \n", " capture_output=True, text=True, timeout=5)\n", " if result.returncode == 0:\n", " print(' ✅ Service is already running')\n", " else:\n", " print(' ⚙️ Starting Foundry Local service...')\n", " subprocess.run(['foundry', 'service', 'start'], \n", " capture_output=True, text=True, timeout=30)\n", " time.sleep(5)\n", " print(' ✅ Service started')\n", "except Exception as e:\n", " print(f' ⚠️ Could not verify service: {e}')\n", " print(' 💡 Try manually: foundry service start')\n", "\n", "# Step 2: Load each model with --retain true\n", "print(f'\\n🤖 Step 2: Loading models with retention...')\n", "for i, model in enumerate(REQUIRED_MODELS, 1):\n", " print(f' [{i}/{len(REQUIRED_MODELS)}] Starting {model}...')\n", " try:\n", " subprocess.Popen(['foundry', 'model', 'run', model, '--retain', 'true'],\n", " stdout=subprocess.DEVNULL,\n", " stderr=subprocess.DEVNULL)\n", " print(f' ✅ {model} loading in background')\n", " except Exception as e:\n", " print(f' ❌ Error starting {model}: {e}')\n", "\n", "# Step 3: Verify models are ready\n", "print(f'\\n✅ Step 3: Verifying models (this may take 2-3 minutes)...')\n", "print('=' * 70)\n", "\n", "try:\n", " from workshop_utils import get_client\n", " \n", " ready_models = []\n", " max_attempts = 30\n", " attempt = 0\n", " \n", " while len(ready_models) < len(REQUIRED_MODELS) and attempt < max_attempts:\n", " attempt += 1\n", " print(f'\\n Attempt {attempt}/{max_attempts}...')\n", " \n", " for model in REQUIRED_MODELS:\n", " if model in ready_models:\n", " continue\n", " \n", " try:\n", " manager, client, model_id = get_client(model)\n", " response = client.chat.completions.create(\n", " model=model_id,\n", " messages=[{\"role\": \"user\", \"content\": \"test\"}],\n", " max_tokens=5,\n", " temperature=0\n", " )\n", " \n", " if response and response.choices:\n", " ready_models.append(model)\n", " print(f' ✅ {model} is READY')\n", " \n", " except Exception as e:\n", " error_msg = str(e).lower()\n", " if 'connection' in error_msg or 'timeout' in error_msg:\n", " print(f' ⏳ {model} still loading...')\n", " else:\n", " print(f' ⚠️ {model} error: {str(e)[:60]}...')\n", " \n", " if len(ready_models) == len(REQUIRED_MODELS):\n", " break\n", " \n", " if len(ready_models) < len(REQUIRED_MODELS):\n", " time.sleep(10)\n", " \n", " # Final status\n", " print('\\n' + '=' * 70)\n", " print(f'📦 Final Status: {len(ready_models)}/{len(REQUIRED_MODELS)} models ready')\n", " \n", " for model in REQUIRED_MODELS:\n", " if model in ready_models:\n", " print(f' ✅ {model} - READY (retained in memory)')\n", " else:\n", " print(f' ❌ {model} - NOT READY')\n", " \n", " if len(ready_models) == len(REQUIRED_MODELS):\n", " print('\\n🎉 All models loaded and verified!')\n", " print(' ✅ Ready for intent-based routing')\n", " else:\n", " print(f'\\n⚠️ Some models not ready. Check: foundry model ls')\n", " \n", "except ImportError as e:\n", " print(f'\\n❌ Cannot import workshop_utils: {e}')\n", " print(' 💡 Ensure workshop_utils.py is in ../samples/')\n", "except Exception as e:\n", " print(f'\\n❌ Verification error: {e}')" ] }, { "cell_type": "markdown", "id": "e682909b", "metadata": {}, "source": [ "## 🎯 الخطوة 4: إعداد كشف النوايا وفهرس النماذج\n", "\n", "قم بإعداد نظام التوجيه باستخدام:\n", "- **قواعد النوايا**: أنماط Regex لتصنيف الطلبات\n", "- **فهرس النماذج**: يربط بين قدرات النماذج وفئات النوايا\n", "- **نظام الأولوية**: يحدد اختيار النموذج عند تطابق عدة نماذج\n", "\n", "**مزايا نموذج CPU**:\n", "- ✅ لا يتطلب GPU\n", "- ✅ أداء ثابت\n", "- ✅ استهلاك طاقة أقل\n", "- ✅ استخدام ذاكرة متوقع\n" ] }, { "cell_type": "code", "execution_count": 110, "id": "3620a4fc", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "📋 Active Model Catalog (Hardware-Optimized Aliases)\n", "======================================================================\n", "💡 Using model aliases - Foundry automatically selects CPU variants\n", "\n", " • phi-4-mini\n", " Capabilities: general, summarize, reasoning\n", " Priority: 3\n", "\n", " • qwen2.5-0.5b\n", " Capabilities: classification, fast, general\n", " Priority: 1\n", "\n", " • phi-3.5-mini\n", " Capabilities: code, refactor, technical\n", " Priority: 2\n", "\n", " • qwen2.5-coder-0.5b\n", " Capabilities: code, programming, debug\n", " Priority: 1\n", "\n", "✅ Intent detection and model selection configured\n", "======================================================================\n", "\n", "======================================================================\n", "💡 Using model aliases - Foundry automatically selects CPU variants\n", "\n", " • phi-4-mini\n", " Capabilities: general, summarize, reasoning\n", " Priority: 3\n", "\n", " • qwen2.5-0.5b\n", " Capabilities: classification, fast, general\n", " Priority: 1\n", "\n", " • phi-3.5-mini\n", " Capabilities: code, refactor, technical\n", " Priority: 2\n", "\n", " • qwen2.5-coder-0.5b\n", " Capabilities: code, programming, debug\n", " Priority: 1\n", "\n", "✅ Intent detection and model selection configured\n", "======================================================================\n" ] } ], "source": [ "import re\n", "\n", "# Model capability catalog (maps model aliases to capabilities)\n", "# Use base aliases - Foundry Local will automatically select CPU variants\n", "CATALOG = {\n", " 'phi-4-mini': {\n", " 'capabilities': ['general', 'summarize', 'reasoning'],\n", " 'priority': 3\n", " },\n", " 'qwen2.5-0.5b': {\n", " 'capabilities': ['classification', 'fast', 'general'],\n", " 'priority': 1\n", " },\n", " 'phi-3.5-mini': {\n", " 'capabilities': ['code', 'refactor', 'technical'],\n", " 'priority': 2\n", " },\n", " 'qwen2.5-coder-0.5b': {\n", " 'capabilities': ['code', 'programming', 'debug'],\n", " 'priority': 1\n", " }\n", "}\n", "\n", "# Filter to only include models recommended for this system\n", "CATALOG = {k: v for k, v in CATALOG.items() if k in model_aliases}\n", "\n", "print('📋 Active Model Catalog (Hardware-Optimized Aliases)')\n", "print('=' * 70)\n", "print('💡 Using model aliases - Foundry automatically selects CPU variants')\n", "print()\n", "for model, info in CATALOG.items():\n", " caps = ', '.join(info['capabilities'])\n", " print(f' • {model}')\n", " print(f' Capabilities: {caps}')\n", " print(f' Priority: {info[\"priority\"]}')\n", " print()\n", "\n", "# Intent detection rules (regex pattern -> intent label)\n", "INTENT_RULES = [\n", " (re.compile(r'code|refactor|function|debug|program', re.I), 'code'),\n", " (re.compile(r'summar|abstract|tl;?dr|brief', re.I), 'summarize'),\n", " (re.compile(r'classif|categor|label|sentiment', re.I), 'classification'),\n", " (re.compile(r'explain|teach|describe', re.I), 'general'),\n", "]\n", "\n", "def detect_intent(prompt: str) -> str:\n", " \"\"\"Detect intent from prompt using regex patterns.\n", " \n", " Args:\n", " prompt: User input text\n", " \n", " Returns:\n", " Intent label: 'code', 'summarize', 'classification', or 'general'\n", " \"\"\"\n", " for pattern, intent in INTENT_RULES:\n", " if pattern.search(prompt):\n", " return intent\n", " return 'general'\n", "\n", "def pick_model(intent: str) -> str:\n", " \"\"\"Select best model for intent based on capabilities and priority.\n", " \n", " Args:\n", " intent: Detected intent category\n", " \n", " Returns:\n", " Model alias string, or first available model if no match\n", " \"\"\"\n", " candidates = [\n", " (alias, info['priority']) \n", " for alias, info in CATALOG.items() \n", " if intent in info['capabilities']\n", " ]\n", " \n", " if candidates:\n", " # Sort by priority (higher = better)\n", " candidates.sort(key=lambda x: x[1], reverse=True)\n", " return candidates[0][0]\n", " \n", " # Fallback to first available model\n", " return list(CATALOG.keys())[0] if CATALOG else None\n", "\n", "print('✅ Intent detection and model selection configured')\n", "print('=' * 70)" ] }, { "cell_type": "markdown", "id": "5fbb6d09", "metadata": {}, "source": [ "## 🧪 الخطوة 5: اختبار اكتشاف النوايا\n", "\n", "تحقق من أن نظام اكتشاف النوايا يصنف بشكل صحيح الأنواع المختلفة من المطالبات.\n" ] }, { "cell_type": "code", "execution_count": 111, "id": "0fd85468", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🧪 Testing Intent Detection\n", "======================================================================\n", "\n", "Prompt: Refactor this Python function for better readabili...\n", " Intent: code → Model: phi-3.5-mini\n", "\n", "Prompt: Summarize the key points of this article...\n", " Intent: summarize → Model: phi-4-mini\n", "\n", "Prompt: Classify this customer feedback as positive or neg...\n", " Intent: classification → Model: qwen2.5-0.5b\n", "\n", "Prompt: Explain how edge AI differs from cloud AI...\n", " Intent: general → Model: phi-4-mini\n", "\n", "Prompt: Write a function to calculate fibonacci numbers...\n", " Intent: code → Model: phi-3.5-mini\n", "\n", "Prompt: Give me a brief overview of small language models...\n", " Intent: summarize → Model: phi-4-mini\n", "\n", "======================================================================\n", "✅ Intent detection working correctly\n" ] } ], "source": [ "# Test intent detection with sample prompts\n", "test_prompts = [\n", " 'Refactor this Python function for better readability',\n", " 'Summarize the key points of this article',\n", " 'Classify this customer feedback as positive or negative',\n", " 'Explain how edge AI differs from cloud AI',\n", " 'Write a function to calculate fibonacci numbers',\n", " 'Give me a brief overview of small language models'\n", "]\n", "\n", "print('🧪 Testing Intent Detection')\n", "print('=' * 70)\n", "\n", "for prompt in test_prompts:\n", " intent = detect_intent(prompt)\n", " model = pick_model(intent)\n", " print(f'\\nPrompt: {prompt[:50]}...')\n", " print(f' Intent: {intent:15s} → Model: {model}')\n", "\n", "print('\\n' + '=' * 70)\n", "print('✅ Intent detection working correctly')" ] }, { "cell_type": "markdown", "id": "9ae6a08b", "metadata": {}, "source": [ "## 🚀 الخطوة 6: تنفيذ وظيفة التوجيه\n", "\n", "قم بإنشاء وظيفة التوجيه الرئيسية التي:\n", "1. تكتشف النية من النص المدخل\n", "2. تختار النموذج الأمثل\n", "3. تنفذ الطلب عبر Foundry Local SDK\n", "4. تتبع استخدام الرموز والأخطاء\n", "\n", "**تستخدم نمط workshop_utils**:\n", "- إعادة المحاولة التلقائية مع زيادة زمنية تدريجية\n", "- واجهة برمجة تطبيقات متوافقة مع OpenAI\n", "- تتبع الرموز ومعالجة الأخطاء\n" ] }, { "cell_type": "code", "execution_count": 112, "id": "24cc251d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Routing function ready\n", " Using Foundry Local SDK via workshop_utils\n", " Token tracking: Enabled\n", " Retry logic: Automatic with exponential backoff\n" ] } ], "source": [ "import os\n", "from workshop_utils import chat_once\n", "\n", "# Fix RETRY_BACKOFF environment variable if it has comments\n", "if 'RETRY_BACKOFF' in os.environ:\n", " retry_val = os.environ['RETRY_BACKOFF'].strip().split()[0]\n", " try:\n", " float(retry_val)\n", " os.environ['RETRY_BACKOFF'] = retry_val\n", " except ValueError:\n", " os.environ['RETRY_BACKOFF'] = '1.0'\n", "\n", "def route(prompt: str, max_tokens: int = 200, temperature: float = 0.7):\n", " \"\"\"Route prompt to appropriate model based on intent.\n", " \n", " Pipeline:\n", " 1. Detect intent using regex patterns\n", " 2. Select best model by capability + priority\n", " 3. Execute via Foundry Local SDK\n", " \n", " Args:\n", " prompt: User input text\n", " max_tokens: Maximum tokens in response\n", " temperature: Sampling temperature (0-1)\n", " \n", " Returns:\n", " Dict with: intent, model, output, tokens, usage, error\n", " \"\"\"\n", " intent = detect_intent(prompt)\n", " model_alias = pick_model(intent)\n", " \n", " if not model_alias:\n", " return {\n", " 'intent': intent,\n", " 'model': None,\n", " 'output': '',\n", " 'tokens': None,\n", " 'usage': {},\n", " 'error': 'No suitable model found'\n", " }\n", " \n", " try:\n", " # Call Foundry Local via workshop_utils\n", " text, usage = chat_once(\n", " model_alias,\n", " messages=[{\"role\": \"user\", \"content\": prompt}],\n", " max_tokens=max_tokens,\n", " temperature=temperature\n", " )\n", " \n", " # Extract token information\n", " usage_info = {}\n", " if usage:\n", " usage_info['prompt_tokens'] = getattr(usage, 'prompt_tokens', None)\n", " usage_info['completion_tokens'] = getattr(usage, 'completion_tokens', None)\n", " usage_info['total_tokens'] = getattr(usage, 'total_tokens', None)\n", " \n", " # Estimate if not provided\n", " if not usage_info.get('total_tokens'):\n", " est_prompt = len(prompt) // 4\n", " est_completion = len(text or '') // 4\n", " usage_info['estimated_tokens'] = est_prompt + est_completion\n", " \n", " return {\n", " 'intent': intent,\n", " 'model': model_alias,\n", " 'output': (text or '').strip(),\n", " 'tokens': usage_info.get('total_tokens') or usage_info.get('estimated_tokens'),\n", " 'usage': usage_info,\n", " 'error': None\n", " }\n", " \n", " except Exception as e:\n", " return {\n", " 'intent': intent,\n", " 'model': model_alias,\n", " 'output': '',\n", " 'tokens': None,\n", " 'usage': {},\n", " 'error': f'{type(e).__name__}: {str(e)}'\n", " }\n", "\n", "print('✅ Routing function ready')\n", "print(' Using Foundry Local SDK via workshop_utils')\n", "print(' Token tracking: Enabled')\n", "print(' Retry logic: Automatic with exponential backoff')" ] }, { "cell_type": "markdown", "id": "00a5c915", "metadata": {}, "source": [ "## 🎯 الخطوة 7: تشغيل اختبارات التوجيه\n", "\n", "اختبر نظام التوجيه الكامل باستخدام مجموعة متنوعة من المطالبات لتوضيح:\n", "- الكشف التلقائي عن النوايا\n", "- اختيار النموذج الذكي\n", "- التوجيه متعدد النماذج مع الاحتفاظ بالنماذج\n", "- تتبع الرموز ومقاييس الأداء\n" ] }, { "cell_type": "code", "execution_count": null, "id": "85c46ef4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🎯 Running Intent-Based Routing Tests\n", "================================================================================\n", "\n", "[1/6] Testing prompt...\n", "Prompt: Refactor this Python function to make it more efficient and readable\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " Expected Intent: code\n", " Detected Intent: code ✅\n", " Selected Model: phi-3.5-mini\n", " ✅ Response: To refactor a Python function for efficiency and readability, I would need to see the specific funct...\n", " 📊 Tokens: ~158 (estimated)\n", "\n", "[2/6] Testing prompt...\n", "Prompt: Summarize the key benefits of using small language models at the edge\n", " Expected Intent: summarize\n", " Detected Intent: summarize ✅\n", " Selected Model: phi-4-mini\n", " ❌ Error: APIConnectionError: Connection error.\n", "\n", "[3/6] Testing prompt...\n", "Prompt: Classify this user feedback: The app is slow but the UI looks great\n", " Expected Intent: classification\n", " Detected Intent: classification ✅\n", " Selected Model: qwen2.5-0.5b\n", " ❌ Error: APIConnectionError: Connection error.\n", "\n", "[4/6] Testing prompt...\n", "Prompt: Explain the difference between local and cloud inference\n", " Expected Intent: general\n", " Detected Intent: general ✅\n", " Selected Model: phi-4-mini\n", " ❌ Error: APIConnectionError: Connection error.\n", "\n", "[5/6] Testing prompt...\n", "Prompt: Write a Python function to calculate the Fibonacci sequence\n" ] } ], "source": [ "# Test prompts covering all intent categories\n", "test_cases = [\n", " {\n", " 'prompt': 'Refactor this Python function to make it more efficient and readable',\n", " 'expected_intent': 'code'\n", " },\n", " {\n", " 'prompt': 'Summarize the key benefits of using small language models at the edge',\n", " 'expected_intent': 'summarize'\n", " },\n", " {\n", " 'prompt': 'Classify this user feedback: The app is slow but the UI looks great',\n", " 'expected_intent': 'classification'\n", " },\n", " {\n", " 'prompt': 'Explain the difference between local and cloud inference',\n", " 'expected_intent': 'general'\n", " },\n", " {\n", " 'prompt': 'Write a Python function to calculate the Fibonacci sequence',\n", " 'expected_intent': 'code'\n", " },\n", " {\n", " 'prompt': 'Give me a brief overview of the Phi model family',\n", " 'expected_intent': 'summarize'\n", " }\n", "]\n", "\n", "print('🎯 Running Intent-Based Routing Tests')\n", "print('=' * 80)\n", "\n", "results = []\n", "for i, test in enumerate(test_cases, 1):\n", " print(f'\\n[{i}/{len(test_cases)}] Testing prompt...')\n", " print(f'Prompt: {test[\"prompt\"]}')\n", " \n", " result = route(test['prompt'], max_tokens=150)\n", " results.append(result)\n", " \n", " print(f' Expected Intent: {test[\"expected_intent\"]}')\n", " print(f' Detected Intent: {result[\"intent\"]} {\"✅\" if result[\"intent\"] == test[\"expected_intent\"] else \"⚠️\"}')\n", " print(f' Selected Model: {result[\"model\"]}')\n", " \n", " if result['error']:\n", " print(f' ❌ Error: {result[\"error\"]}')\n", " else:\n", " output_preview = result['output'][:100] + '...' if len(result['output']) > 100 else result['output']\n", " print(f' ✅ Response: {output_preview}')\n", " \n", " tokens = result.get('tokens', 0)\n", " if tokens:\n", " usage = result.get('usage', {})\n", " if 'estimated_tokens' in usage:\n", " print(f' 📊 Tokens: ~{tokens} (estimated)')\n", " else:\n", " print(f' 📊 Tokens: {tokens}')\n", "\n", "# Summary statistics\n", "print('\\n' + '=' * 80)\n", "print('📊 ROUTING SUMMARY')\n", "print('=' * 80)\n", "\n", "success_count = sum(1 for r in results if not r['error'])\n", "total_tokens = sum(r.get('tokens', 0) or 0 for r in results if not r['error'])\n", "intent_accuracy = sum(1 for i, r in enumerate(results) if r['intent'] == test_cases[i]['expected_intent'])\n", "\n", "print(f'Total Prompts: {len(results)}')\n", "print(f'✅ Successful: {success_count}/{len(results)}')\n", "print(f'❌ Failed: {len(results) - success_count}')\n", "print(f'🎯 Intent Accuracy: {intent_accuracy}/{len(results)} ({intent_accuracy/len(results)*100:.1f}%)')\n", "print(f'📊 Total Tokens Used: {total_tokens}')\n", "\n", "# Model usage distribution\n", "print('\\n📋 Model Usage Distribution:')\n", "model_counts = {}\n", "for r in results:\n", " if r['model']:\n", " model_counts[r['model']] = model_counts.get(r['model'], 0) + 1\n", "\n", "for model, count in sorted(model_counts.items(), key=lambda x: x[1], reverse=True):\n", " percentage = (count / len(results)) * 100\n", " print(f' • {model}: {count} requests ({percentage:.1f}%)')\n", "\n", "if success_count == len(results):\n", " print('\\n🎉 All routing tests passed successfully!')\n", "else:\n", " print(f'\\n⚠️ {len(results) - success_count} test(s) failed')\n", " print(' Check Foundry Local service: foundry service status')\n", " print(' Verify models loaded: foundry model ls')\n", "\n", "print('=' * 80)" ] }, { "cell_type": "markdown", "id": "4764811e", "metadata": {}, "source": [ "## 🔧 الخطوة 8: الاختبار التفاعلي\n", "\n", "جرّب إدخال التعليمات الخاصة بك لترى نظام التوجيه قيد العمل!\n" ] }, { "cell_type": "code", "execution_count": null, "id": "3f8fdd51", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "🎯 Interactive Routing Test\n", "================================================================================\n", "Your prompt: Explain how model quantization reduces memory usage\n", "\n", "Detected Intent: general\n", "Selected Model: phi-4-mini\n", "\n", "✅ Response:\n", "--------------------------------------------------------------------------------\n", "Model quantization is a technique used to reduce the memory footprint of a machine learning model, particularly deep learning models. It works by converting the high-precision weights of a neural network, typically represented as 32-bit floating-point numbers, into lower-precision representations, such as 8-bit integers or even binary values.\n", "\n", "\n", "The primary reason for quantization is to decrease the amount of memory required to store the model's parameters. Since floating-point numbers take up more space than integers, by quantizing the weights, we can significantly reduce the model's size. This reduction in size not only saves memory but also can lead to faster computation during inference, as integer operations are generally faster than floating-point operations on many hardware platforms.\n", "\n", "\n", "However, quantization can introduce some loss of accuracy because the lower precision representation may not capture the full range of values that the floating-point representation can. To mitigate this, techniques such as quantization-aware training can be used, where the model is trained with quantization in mind,\n", "--------------------------------------------------------------------------------\n", "\n", "📊 Tokens used: 292\n", "\n", "💡 Try different prompts to test routing behavior!\n", "Detected Intent: general\n", "Selected Model: phi-4-mini\n", "\n", "✅ Response:\n", "--------------------------------------------------------------------------------\n", "Model quantization is a technique used to reduce the memory footprint of a machine learning model, particularly deep learning models. It works by converting the high-precision weights of a neural network, typically represented as 32-bit floating-point numbers, into lower-precision representations, such as 8-bit integers or even binary values.\n", "\n", "\n", "The primary reason for quantization is to decrease the amount of memory required to store the model's parameters. Since floating-point numbers take up more space than integers, by quantizing the weights, we can significantly reduce the model's size. This reduction in size not only saves memory but also can lead to faster computation during inference, as integer operations are generally faster than floating-point operations on many hardware platforms.\n", "\n", "\n", "However, quantization can introduce some loss of accuracy because the lower precision representation may not capture the full range of values that the floating-point representation can. To mitigate this, techniques such as quantization-aware training can be used, where the model is trained with quantization in mind,\n", "--------------------------------------------------------------------------------\n", "\n", "📊 Tokens used: 292\n", "\n", "💡 Try different prompts to test routing behavior!\n" ] } ], "source": [ "# Interactive testing - modify the prompt and run this cell\n", "custom_prompt = \"Explain how model quantization reduces memory usage\"\n", "\n", "print('🎯 Interactive Routing Test')\n", "print('=' * 80)\n", "print(f'Your prompt: {custom_prompt}')\n", "print()\n", "\n", "result = route(custom_prompt, max_tokens=200)\n", "\n", "print(f'Detected Intent: {result[\"intent\"]}')\n", "print(f'Selected Model: {result[\"model\"]}')\n", "print()\n", "\n", "if result['error']:\n", " print(f'❌ Error: {result[\"error\"]}')\n", "else:\n", " print('✅ Response:')\n", " print('-' * 80)\n", " print(result['output'])\n", " print('-' * 80)\n", " \n", " if result['tokens']:\n", " print(f'\\n📊 Tokens used: {result[\"tokens\"]}')\n", "\n", "print('\\n💡 Try different prompts to test routing behavior!')" ] }, { "cell_type": "markdown", "id": "1c17226c", "metadata": {}, "source": [ "## 📊 الخطوة 9: تحليل الأداء\n", "\n", "قم بتحليل أداء نظام التوجيه واستخدام النموذج.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "805c688c", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "⚡ Performance Benchmark\n", "================================================================================\n", "\n", "Prompt: Write a hello world function...\n", " Model: phi-3.5-mini\n", " Time: 3.31s\n", " Tokens: 60\n", "\n", "Prompt: Write a hello world function...\n", " Model: phi-3.5-mini\n", " Time: 3.31s\n", " Tokens: 60\n", "\n", "Prompt: Summarize: AI at the edge is powerful...\n", " Model: phi-4-mini\n", " Time: 49.67s\n", " Tokens: 84\n", "\n", "Prompt: Summarize: AI at the edge is powerful...\n", " Model: phi-4-mini\n", " Time: 49.67s\n", " Tokens: 84\n", "\n", "Prompt: Classify: Good product...\n", " Model: qwen2.5-0.5b\n", " Time: 7.21s\n", " Tokens: 69\n", "\n", "Prompt: Classify: Good product...\n", " Model: qwen2.5-0.5b\n", " Time: 7.21s\n", " Tokens: 69\n", "\n", "Prompt: Explain edge computing...\n", " Model: phi-4-mini\n", " Time: 49.67s\n", " Tokens: 72\n", "\n", "================================================================================\n", "📊 Performance Statistics:\n", " Average response time: 27.46s\n", " Fastest response: 3.31s\n", " Slowest response: 49.67s\n", "\n", "💡 Note: First request may be slower due to model initialization\n", "================================================================================\n", "\n", "Prompt: Explain edge computing...\n", " Model: phi-4-mini\n", " Time: 49.67s\n", " Tokens: 72\n", "\n", "================================================================================\n", "📊 Performance Statistics:\n", " Average response time: 27.46s\n", " Fastest response: 3.31s\n", " Slowest response: 49.67s\n", "\n", "💡 Note: First request may be slower due to model initialization\n", "================================================================================\n" ] } ], "source": [ "import time\n", "\n", "# Performance benchmark\n", "benchmark_prompts = [\n", " 'Write a hello world function',\n", " 'Summarize: AI at the edge is powerful',\n", " 'Classify: Good product',\n", " 'Explain edge computing'\n", "]\n", "\n", "print('⚡ Performance Benchmark')\n", "print('=' * 80)\n", "\n", "timings = []\n", "for prompt in benchmark_prompts:\n", " start = time.time()\n", " result = route(prompt, max_tokens=50)\n", " duration = time.time() - start\n", " timings.append(duration)\n", " \n", " print(f'\\nPrompt: {prompt[:40]}...')\n", " print(f' Model: {result[\"model\"]}')\n", " print(f' Time: {duration:.2f}s')\n", " if result.get('tokens'):\n", " print(f' Tokens: {result[\"tokens\"]}')\n", "\n", "print('\\n' + '=' * 80)\n", "print('📊 Performance Statistics:')\n", "print(f' Average response time: {sum(timings)/len(timings):.2f}s')\n", "print(f' Fastest response: {min(timings):.2f}s')\n", "print(f' Slowest response: {max(timings):.2f}s')\n", "print('\\n💡 Note: First request may be slower due to model initialization')\n", "print('=' * 80)" ] }, { "cell_type": "markdown", "id": "e7db64ff", "metadata": {}, "source": [ "## 🎓 النقاط الرئيسية والخطوات التالية\n", "\n", "### ✅ ما تعلمته\n", "\n", "1. **التوجيه بناءً على النوايا**: تصنيف الطلبات تلقائيًا وتوجيهها إلى النماذج المتخصصة \n", "2. **اختيار واعٍ للذاكرة**: اختيار نماذج CPU بناءً على ذاكرة النظام المتاحة \n", "3. **الاحتفاظ بالنماذج المتعددة**: استخدام `--retain true` للحفاظ على تحميل نماذج متعددة \n", "4. **أنماط الإنتاج**: منطق إعادة المحاولة، معالجة الأخطاء، وتتبع الرموز \n", "5. **تحسين أداء CPU**: النشر بكفاءة دون الحاجة إلى GPU \n", "\n", "### 🚀 أفكار للتجربة\n", "\n", "1. **إضافة نوايا مخصصة**: \n", " ```python\n", " INTENT_RULES.append(\n", " (re.compile(r'translate|convert', re.I), 'translation')\n", " )\n", " ```\n", " \n", "2. **تحميل نماذج إضافية**: \n", " ```bash\n", " foundry model run llama-3.2-1b-cpu --retain true\n", " ```\n", " \n", "3. **ضبط اختيار النموذج**: \n", " - تعديل قيم الأولوية في CATALOG \n", " - إضافة المزيد من علامات القدرات \n", " - تنفيذ استراتيجيات احتياطية \n", "\n", "4. **مراقبة الأداء**: \n", " ```python\n", " import psutil\n", " print(f\"Memory: {psutil.virtual_memory().percent}%\")\n", " ```\n", " \n", "\n", "### 📚 موارد إضافية\n", "\n", "- **Foundry Local SDK**: https://github.com/microsoft/Foundry-Local \n", "- **عينات الورشة**: ../samples/ \n", "- **دورة الذكاء الاصطناعي الطرفي**: ../../Module08/ \n", "\n", "### 💡 أفضل الممارسات\n", "\n", "✅ استخدم نماذج CPU للحصول على أداء متسق عبر المنصات المختلفة \n", "✅ تحقق دائمًا من ذاكرة النظام قبل تحميل نماذج متعددة \n", "✅ استخدم `--retain true` في سيناريوهات التوجيه \n", "✅ قم بتنفيذ معالجة الأخطاء وإعادة المحاولة بشكل صحيح \n", "✅ تتبع استخدام الرموز لتحسين التكلفة والأداء \n", "\n", "---\n", "\n", "**🎉 تهانينا!** لقد قمت ببناء موجه نماذج جاهز للإنتاج يعتمد على النوايا باستخدام Foundry Local SDK مع نماذج محسّنة لـ CPU!\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n\n\n**إخلاء المسؤولية**: \nتمت ترجمة هذا المستند باستخدام خدمة الترجمة بالذكاء الاصطناعي [Co-op Translator](https://github.com/Azure/co-op-translator). بينما نسعى لتحقيق الدقة، يرجى العلم أن الترجمات الآلية قد تحتوي على أخطاء أو عدم دقة. يجب اعتبار المستند الأصلي بلغته الأصلية المصدر الموثوق. للحصول على معلومات حاسمة، يُوصى بالترجمة البشرية الاحترافية. نحن غير مسؤولين عن أي سوء فهم أو تفسيرات خاطئة ناتجة عن استخدام هذه الترجمة.\n\n" ] } ], "metadata": { "kernelspec": { "display_name": "demo", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.15" }, "coopTranslator": { "original_hash": "2fbaf6df2c479cfd0bb85547ef819dff", "translation_date": "2025-11-18T16:43:38+00:00", "source_file": "Workshop/notebooks/session06_models_router.ipynb", "language_code": "ar" } }, "nbformat": 4, "nbformat_minor": 5 }