{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "af028554",
   "metadata": {},
   "source": [
    "# Модел за маршрутизация на базата на намерения с Foundry Local SDK\n",
    "\n",
    "**Система за маршрутизация на множество модели, оптимизирана за CPU**\n",
    "\n",
    "Този тетрадка демонстрира интелигентна система за маршрутизация, която автоматично избира най-добрия малък езиков модел въз основа на намерението на потребителя. Идеално за сценарии на внедряване на крайни устройства, където искате ефективно да използвате множество специализирани модели.\n",
    "\n",
    "## 🎯 Какво ще научите\n",
    "\n",
    "- **Откриване на намерения**: Автоматично класифициране на запитвания (код, обобщение, класификация, общо)\n",
    "- **Интелигентен избор на модел**: Маршрутизация към най-способния модел за всяка задача\n",
    "- **Оптимизация за CPU**: Модели с ефективно използване на паметта, които работят на всякакъв хардуер\n",
    "- **Управление на множество модели**: Поддържайте заредени множество модели с `--retain true`\n",
    "- **Производствени модели**: Логика за повторение, обработка на грешки и проследяване на токени\n",
    "\n",
    "## 📋 Преглед на сценария\n",
    "\n",
    "Този модел демонстрира:\n",
    "\n",
    "1. **Откриване на намерения**: Класифициране на всяко потребителско запитване (код, обобщение, класификация или общо)\n",
    "2. **Избор на модел**: Автоматично избиране на най-подходящия малък езиков модел въз основа на възможностите\n",
    "3. **Локално изпълнение**: Маршрутизация към модели, работещи локално чрез услугата Foundry Local\n",
    "4. **Обединен интерфейс**: Една точка за чат, която маршрутизира към множество специализирани модели\n",
    "\n",
    "**Идеално за**: Внедряване на крайни устройства с множество специализирани модели, където искате интелигентно маршрутизиране на заявки без ръчен избор на модел.\n",
    "\n",
    "## 🔧 Предварителни изисквания\n",
    "\n",
    "- **Инсталиран Foundry Local** и работеща услуга\n",
    "- **Python 3.8+** с pip\n",
    "- **8GB+ RAM** (препоръчително 16GB+ за множество модели)\n",
    "- **workshop_utils** модул (в ../samples/)\n",
    "\n",
    "## 🚀 Бърз старт\n",
    "\n",
    "Тетрадката ще:\n",
    "1. Открие паметта на вашата система\n",
    "2. Препоръча подходящи CPU модели\n",
    "3. Автоматично зареди модели с `--retain true`\n",
    "4. Провери дали всички модели са готови\n",
    "5. Маршрутизира тестови запитвания към специализирани модели\n",
    "\n",
    "**Очаквано време за настройка**: 5-7 минути (включва зареждане на модели)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4aa55f0",
   "metadata": {},
   "source": [
    "## 📦 Стъпка 1: Инсталиране на зависимости\n",
    "\n",
    "Инсталирайте официалния Foundry Local SDK и необходимите библиотеки:\n",
    "\n",
    "- **foundry-local-sdk**: Официален Python SDK за локално управление на модели\n",
    "- **openai**: API, съвместим с OpenAI, за завършване на чатове\n",
    "- **psutil**: Откриване и мониторинг на системната памет\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "2929c9f5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install core dependencies\n",
    "!pip install -q foundry-local-sdk openai psutil"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "990799e7",
   "metadata": {},
   "source": [
    "## 💻 Стъпка 2: Откриване на системната памет\n",
    "\n",
    "Открийте наличната системна памет, за да определите кои модели CPU могат да работят ефективно. Това гарантира оптимален избор на модел за вашия хардуер.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "ff58f1ee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "🖥️  System Memory Information\n",
      "======================================================================\n",
      "Total Memory:     63.30 GB\n",
      "Available Memory: 16.19 GB\n",
      "\n",
      "✅ High Memory System (32GB+)\n",
      "   Can run 3-4 models simultaneously\n",
      "\n",
      "📋 Recommended Model Aliases for Your System:\n",
      "   • phi-4-mini\n",
      "   • phi-3.5-mini\n",
      "   • qwen2.5-0.5b\n",
      "   • qwen2.5-coder-0.5b\n",
      "\n",
      "💡 About Model Aliases:\n",
      "   ✓ Use base alias (e.g., phi-4-mini, not phi-4-mini-cpu)\n",
      "   ✓ Foundry Local automatically selects CPU variant for your hardware\n",
      "   ✓ No GPU required - optimized for CPU inference\n",
      "   ✓ Predictable memory usage and consistent performance\n",
      "======================================================================\n"
     ]
    }
   ],
   "source": [
    "import psutil\n",
    "\n",
    "# Get system memory information\n",
    "total_memory_gb = psutil.virtual_memory().total / (1024**3)\n",
    "available_memory_gb = psutil.virtual_memory().available / (1024**3)\n",
    "\n",
    "print('🖥️  System Memory Information')\n",
    "print('=' * 70)\n",
    "print(f'Total Memory:     {total_memory_gb:.2f} GB')\n",
    "print(f'Available Memory: {available_memory_gb:.2f} GB')\n",
    "print()\n",
    "\n",
    "# Recommend models based on available memory\n",
    "# Using model aliases - Foundry Local will automatically select CPU variant\n",
    "model_aliases = []\n",
    "\n",
    "if total_memory_gb >= 32:\n",
    "    model_aliases = ['phi-4-mini', 'phi-3.5-mini', 'qwen2.5-0.5b', 'qwen2.5-coder-0.5b']\n",
    "    print('✅ High Memory System (32GB+)')\n",
    "    print('   Can run 3-4 models simultaneously')\n",
    "elif total_memory_gb >= 16:\n",
    "    model_aliases = ['phi-4-mini', 'qwen2.5-0.5b', 'phi-3.5-mini']\n",
    "    print('✅ Medium Memory System (16-32GB)')\n",
    "    print('   Can run 2-3 models simultaneously')\n",
    "elif total_memory_gb >= 8:\n",
    "    model_aliases = ['qwen2.5-0.5b', 'phi-3.5-mini']\n",
    "    print('⚠️  Lower Memory System (8-16GB)')\n",
    "    print('   Recommended: 2 smaller models')\n",
    "else:\n",
    "    model_aliases = ['qwen2.5-0.5b']\n",
    "    print('⚠️  Limited Memory System (<8GB)')\n",
    "    print('   Recommended: Use only smallest model')\n",
    "\n",
    "print()\n",
    "print('📋 Recommended Model Aliases for Your System:')\n",
    "for model in model_aliases:\n",
    "    print(f'   • {model}')\n",
    "\n",
    "print()\n",
    "print('💡 About Model Aliases:')\n",
    "print('   ✓ Use base alias (e.g., phi-4-mini, not phi-4-mini-cpu)')\n",
    "print('   ✓ Foundry Local automatically selects CPU variant for your hardware')\n",
    "print('   ✓ No GPU required - optimized for CPU inference')\n",
    "print('   ✓ Predictable memory usage and consistent performance')\n",
    "print('=' * 70)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69590b94",
   "metadata": {},
   "source": [
    "## 🤖 Стъпка 3: Автоматично зареждане на моделите\n",
    "\n",
    "Тази клетка автоматично:\n",
    "1. Стартира Foundry Local услуга (ако не е активирана)\n",
    "2. Зарежда препоръчаните модели с `--retain true` (запазва няколко модела в паметта)\n",
    "3. Проверява дали всички модели са готови чрез SDK\n",
    "\n",
    "⏱️ **Очаквано време**: 3-5 минути за всички модели\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "543fd976",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "🚀 Automatic Model Loading with SDK Verification\n",
      "======================================================================\n",
      "📋 Loading 3 models: ['phi-4-mini', 'phi-3.5-mini', 'qwen2.5-0.5b']\n",
      "💡 Using model aliases - Foundry will load CPU variants automatically\n",
      "\n",
      "📡 Step 1: Checking Foundry Local service...\n",
      "   ✅ Service is already running\n",
      "\n",
      "🤖 Step 2: Loading models with retention...\n",
      "   [1/3] Starting phi-4-mini...\n",
      "       ✅ phi-4-mini loading in background\n",
      "   [2/3] Starting phi-3.5-mini...\n",
      "       ✅ phi-3.5-mini loading in background\n",
      "   [3/3] Starting qwen2.5-0.5b...\n",
      "       ✅ qwen2.5-0.5b loading in background\n",
      "\n",
      "✅ Step 3: Verifying models (this may take 2-3 minutes)...\n",
      "======================================================================\n",
      "\n",
      "   Attempt 1/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 2/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 3/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 4/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 5/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 6/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 7/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 8/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 9/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 10/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 11/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 12/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 13/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 14/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 15/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 16/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 17/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 18/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 19/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 20/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 21/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 22/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 23/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 24/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 25/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 26/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 27/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 28/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 29/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "   Attempt 30/30...\n",
      "   ⚠️  phi-4-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  phi-3.5-mini error: get_client() takes 1 positional argument but 2 were given...\n",
      "   ⚠️  qwen2.5-0.5b error: get_client() takes 1 positional argument but 2 were given...\n",
      "\n",
      "======================================================================\n",
      "📦 Final Status: 0/3 models ready\n",
      "   ❌ phi-4-mini - NOT READY\n",
      "   ❌ phi-3.5-mini - NOT READY\n",
      "   ❌ qwen2.5-0.5b - NOT READY\n",
      "\n",
      "⚠️  Some models not ready. Check: foundry model ls\n"
     ]
    }
   ],
   "source": [
    "import subprocess\n",
    "import time\n",
    "import sys\n",
    "import os\n",
    "\n",
    "# Add samples directory for workshop_utils (Foundry SDK pattern)\n",
    "sys.path.append(os.path.join('..', 'samples'))\n",
    "\n",
    "print('🚀 Automatic Model Loading with SDK Verification')\n",
    "print('=' * 70)\n",
    "\n",
    "# Use top 3 recommended models (aliases)\n",
    "# Foundry will automatically load CPU variants\n",
    "REQUIRED_MODELS = model_aliases[:3]\n",
    "print(f'📋 Loading {len(REQUIRED_MODELS)} models: {REQUIRED_MODELS}')\n",
    "print('💡 Using model aliases - Foundry will load CPU variants automatically')\n",
    "print()\n",
    "\n",
    "# Step 1: Ensure Foundry Local service is running\n",
    "print('📡 Step 1: Checking Foundry Local service...')\n",
    "try:\n",
    "    result = subprocess.run(['foundry', 'service', 'status'], \n",
    "                          capture_output=True, text=True, timeout=5)\n",
    "    if result.returncode == 0:\n",
    "        print('   ✅ Service is already running')\n",
    "    else:\n",
    "        print('   ⚙️  Starting Foundry Local service...')\n",
    "        subprocess.run(['foundry', 'service', 'start'], \n",
    "                      capture_output=True, text=True, timeout=30)\n",
    "        time.sleep(5)\n",
    "        print('   ✅ Service started')\n",
    "except Exception as e:\n",
    "    print(f'   ⚠️  Could not verify service: {e}')\n",
    "    print('   💡 Try manually: foundry service start')\n",
    "\n",
    "# Step 2: Load each model with --retain true\n",
    "print(f'\\n🤖 Step 2: Loading models with retention...')\n",
    "for i, model in enumerate(REQUIRED_MODELS, 1):\n",
    "    print(f'   [{i}/{len(REQUIRED_MODELS)}] Starting {model}...')\n",
    "    try:\n",
    "        subprocess.Popen(['foundry', 'model', 'run', model, '--retain', 'true'],\n",
    "                        stdout=subprocess.DEVNULL,\n",
    "                        stderr=subprocess.DEVNULL)\n",
    "        print(f'       ✅ {model} loading in background')\n",
    "    except Exception as e:\n",
    "        print(f'       ❌ Error starting {model}: {e}')\n",
    "\n",
    "# Step 3: Verify models are ready\n",
    "print(f'\\n✅ Step 3: Verifying models (this may take 2-3 minutes)...')\n",
    "print('=' * 70)\n",
    "\n",
    "try:\n",
    "    from workshop_utils import get_client\n",
    "    \n",
    "    ready_models = []\n",
    "    max_attempts = 30\n",
    "    attempt = 0\n",
    "    \n",
    "    while len(ready_models) < len(REQUIRED_MODELS) and attempt < max_attempts:\n",
    "        attempt += 1\n",
    "        print(f'\\n   Attempt {attempt}/{max_attempts}...')\n",
    "        \n",
    "        for model in REQUIRED_MODELS:\n",
    "            if model in ready_models:\n",
    "                continue\n",
    "                \n",
    "            try:\n",
    "                manager, client, model_id = get_client(model)\n",
    "                response = client.chat.completions.create(\n",
    "                    model=model_id,\n",
    "                    messages=[{\"role\": \"user\", \"content\": \"test\"}],\n",
    "                    max_tokens=5,\n",
    "                    temperature=0\n",
    "                )\n",
    "                \n",
    "                if response and response.choices:\n",
    "                    ready_models.append(model)\n",
    "                    print(f'   ✅ {model} is READY')\n",
    "                    \n",
    "            except Exception as e:\n",
    "                error_msg = str(e).lower()\n",
    "                if 'connection' in error_msg or 'timeout' in error_msg:\n",
    "                    print(f'   ⏳ {model} still loading...')\n",
    "                else:\n",
    "                    print(f'   ⚠️  {model} error: {str(e)[:60]}...')\n",
    "        \n",
    "        if len(ready_models) == len(REQUIRED_MODELS):\n",
    "            break\n",
    "            \n",
    "        if len(ready_models) < len(REQUIRED_MODELS):\n",
    "            time.sleep(10)\n",
    "    \n",
    "    # Final status\n",
    "    print('\\n' + '=' * 70)\n",
    "    print(f'📦 Final Status: {len(ready_models)}/{len(REQUIRED_MODELS)} models ready')\n",
    "    \n",
    "    for model in REQUIRED_MODELS:\n",
    "        if model in ready_models:\n",
    "            print(f'   ✅ {model} - READY (retained in memory)')\n",
    "        else:\n",
    "            print(f'   ❌ {model} - NOT READY')\n",
    "    \n",
    "    if len(ready_models) == len(REQUIRED_MODELS):\n",
    "        print('\\n🎉 All models loaded and verified!')\n",
    "        print('   ✅ Ready for intent-based routing')\n",
    "    else:\n",
    "        print(f'\\n⚠️  Some models not ready. Check: foundry model ls')\n",
    "        \n",
    "except ImportError as e:\n",
    "    print(f'\\n❌ Cannot import workshop_utils: {e}')\n",
    "    print('   💡 Ensure workshop_utils.py is in ../samples/')\n",
    "except Exception as e:\n",
    "    print(f'\\n❌ Verification error: {e}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e682909b",
   "metadata": {},
   "source": [
    "## 🎯 Стъпка 4: Конфигуриране на откриване на намерения и каталог на модели\n",
    "\n",
    "Настройте системата за маршрутизация с:\n",
    "- **Правила за намерения**: Regex шаблони за класифициране на заявки\n",
    "- **Каталог на модели**: Свързва възможностите на моделите с категории намерения\n",
    "- **Система за приоритети**: Определя избора на модел, когато няколко модела съвпадат\n",
    "\n",
    "**Предимства на CPU моделите**:\n",
    "- ✅ Не изисква GPU\n",
    "- ✅ Постоянна производителност\n",
    "- ✅ По-ниска консумация на енергия\n",
    "- ✅ Предвидима употреба на паметта\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "id": "3620a4fc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "📋 Active Model Catalog (Hardware-Optimized Aliases)\n",
      "======================================================================\n",
      "💡 Using model aliases - Foundry automatically selects CPU variants\n",
      "\n",
      "   • phi-4-mini\n",
      "     Capabilities: general, summarize, reasoning\n",
      "     Priority: 3\n",
      "\n",
      "   • qwen2.5-0.5b\n",
      "     Capabilities: classification, fast, general\n",
      "     Priority: 1\n",
      "\n",
      "   • phi-3.5-mini\n",
      "     Capabilities: code, refactor, technical\n",
      "     Priority: 2\n",
      "\n",
      "   • qwen2.5-coder-0.5b\n",
      "     Capabilities: code, programming, debug\n",
      "     Priority: 1\n",
      "\n",
      "✅ Intent detection and model selection configured\n",
      "======================================================================\n",
      "\n",
      "======================================================================\n",
      "💡 Using model aliases - Foundry automatically selects CPU variants\n",
      "\n",
      "   • phi-4-mini\n",
      "     Capabilities: general, summarize, reasoning\n",
      "     Priority: 3\n",
      "\n",
      "   • qwen2.5-0.5b\n",
      "     Capabilities: classification, fast, general\n",
      "     Priority: 1\n",
      "\n",
      "   • phi-3.5-mini\n",
      "     Capabilities: code, refactor, technical\n",
      "     Priority: 2\n",
      "\n",
      "   • qwen2.5-coder-0.5b\n",
      "     Capabilities: code, programming, debug\n",
      "     Priority: 1\n",
      "\n",
      "✅ Intent detection and model selection configured\n",
      "======================================================================\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "# Model capability catalog (maps model aliases to capabilities)\n",
    "# Use base aliases - Foundry Local will automatically select CPU variants\n",
    "CATALOG = {\n",
    "    'phi-4-mini': {\n",
    "        'capabilities': ['general', 'summarize', 'reasoning'],\n",
    "        'priority': 3\n",
    "    },\n",
    "    'qwen2.5-0.5b': {\n",
    "        'capabilities': ['classification', 'fast', 'general'],\n",
    "        'priority': 1\n",
    "    },\n",
    "    'phi-3.5-mini': {\n",
    "        'capabilities': ['code', 'refactor', 'technical'],\n",
    "        'priority': 2\n",
    "    },\n",
    "    'qwen2.5-coder-0.5b': {\n",
    "        'capabilities': ['code', 'programming', 'debug'],\n",
    "        'priority': 1\n",
    "    }\n",
    "}\n",
    "\n",
    "# Filter to only include models recommended for this system\n",
    "CATALOG = {k: v for k, v in CATALOG.items() if k in model_aliases}\n",
    "\n",
    "print('📋 Active Model Catalog (Hardware-Optimized Aliases)')\n",
    "print('=' * 70)\n",
    "print('💡 Using model aliases - Foundry automatically selects CPU variants')\n",
    "print()\n",
    "for model, info in CATALOG.items():\n",
    "    caps = ', '.join(info['capabilities'])\n",
    "    print(f'   • {model}')\n",
    "    print(f'     Capabilities: {caps}')\n",
    "    print(f'     Priority: {info[\"priority\"]}')\n",
    "    print()\n",
    "\n",
    "# Intent detection rules (regex pattern -> intent label)\n",
    "INTENT_RULES = [\n",
    "    (re.compile(r'code|refactor|function|debug|program', re.I), 'code'),\n",
    "    (re.compile(r'summar|abstract|tl;?dr|brief', re.I), 'summarize'),\n",
    "    (re.compile(r'classif|categor|label|sentiment', re.I), 'classification'),\n",
    "    (re.compile(r'explain|teach|describe', re.I), 'general'),\n",
    "]\n",
    "\n",
    "def detect_intent(prompt: str) -> str:\n",
    "    \"\"\"Detect intent from prompt using regex patterns.\n",
    "    \n",
    "    Args:\n",
    "        prompt: User input text\n",
    "        \n",
    "    Returns:\n",
    "        Intent label: 'code', 'summarize', 'classification', or 'general'\n",
    "    \"\"\"\n",
    "    for pattern, intent in INTENT_RULES:\n",
    "        if pattern.search(prompt):\n",
    "            return intent\n",
    "    return 'general'\n",
    "\n",
    "def pick_model(intent: str) -> str:\n",
    "    \"\"\"Select best model for intent based on capabilities and priority.\n",
    "    \n",
    "    Args:\n",
    "        intent: Detected intent category\n",
    "        \n",
    "    Returns:\n",
    "        Model alias string, or first available model if no match\n",
    "    \"\"\"\n",
    "    candidates = [\n",
    "        (alias, info['priority']) \n",
    "        for alias, info in CATALOG.items() \n",
    "        if intent in info['capabilities']\n",
    "    ]\n",
    "    \n",
    "    if candidates:\n",
    "        # Sort by priority (higher = better)\n",
    "        candidates.sort(key=lambda x: x[1], reverse=True)\n",
    "        return candidates[0][0]\n",
    "    \n",
    "    # Fallback to first available model\n",
    "    return list(CATALOG.keys())[0] if CATALOG else None\n",
    "\n",
    "print('✅ Intent detection and model selection configured')\n",
    "print('=' * 70)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fbb6d09",
   "metadata": {},
   "source": [
    "## 🧪 Стъпка 5: Тест на разпознаването на намерения\n",
    "\n",
    "Уверете се, че системата за разпознаване на намерения правилно класифицира различните видове запитвания.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "0fd85468",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "🧪 Testing Intent Detection\n",
      "======================================================================\n",
      "\n",
      "Prompt: Refactor this Python function for better readabili...\n",
      "   Intent: code            → Model: phi-3.5-mini\n",
      "\n",
      "Prompt: Summarize the key points of this article...\n",
      "   Intent: summarize       → Model: phi-4-mini\n",
      "\n",
      "Prompt: Classify this customer feedback as positive or neg...\n",
      "   Intent: classification  → Model: qwen2.5-0.5b\n",
      "\n",
      "Prompt: Explain how edge AI differs from cloud AI...\n",
      "   Intent: general         → Model: phi-4-mini\n",
      "\n",
      "Prompt: Write a function to calculate fibonacci numbers...\n",
      "   Intent: code            → Model: phi-3.5-mini\n",
      "\n",
      "Prompt: Give me a brief overview of small language models...\n",
      "   Intent: summarize       → Model: phi-4-mini\n",
      "\n",
      "======================================================================\n",
      "✅ Intent detection working correctly\n"
     ]
    }
   ],
   "source": [
    "# Test intent detection with sample prompts\n",
    "test_prompts = [\n",
    "    'Refactor this Python function for better readability',\n",
    "    'Summarize the key points of this article',\n",
    "    'Classify this customer feedback as positive or negative',\n",
    "    'Explain how edge AI differs from cloud AI',\n",
    "    'Write a function to calculate fibonacci numbers',\n",
    "    'Give me a brief overview of small language models'\n",
    "]\n",
    "\n",
    "print('🧪 Testing Intent Detection')\n",
    "print('=' * 70)\n",
    "\n",
    "for prompt in test_prompts:\n",
    "    intent = detect_intent(prompt)\n",
    "    model = pick_model(intent)\n",
    "    print(f'\\nPrompt: {prompt[:50]}...')\n",
    "    print(f'   Intent: {intent:15s} → Model: {model}')\n",
    "\n",
    "print('\\n' + '=' * 70)\n",
    "print('✅ Intent detection working correctly')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ae6a08b",
   "metadata": {},
   "source": [
    "## 🚀 Стъпка 6: Реализиране на функция за маршрутизиране\n",
    "\n",
    "Създайте основната функция за маршрутизиране, която:\n",
    "1. Открива намерението от подадената заявка\n",
    "2. Избира оптималния модел\n",
    "3. Изпълнява заявката чрез Foundry Local SDK\n",
    "4. Следи използването на токени и грешките\n",
    "\n",
    "**Използва модела workshop_utils**:\n",
    "- Автоматично повторение с експоненциално забавяне\n",
    "- Съвместимост с API на OpenAI\n",
    "- Проследяване на токени и обработка на грешки\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "id": "24cc251d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "✅ Routing function ready\n",
      "   Using Foundry Local SDK via workshop_utils\n",
      "   Token tracking: Enabled\n",
      "   Retry logic: Automatic with exponential backoff\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from workshop_utils import chat_once\n",
    "\n",
    "# Fix RETRY_BACKOFF environment variable if it has comments\n",
    "if 'RETRY_BACKOFF' in os.environ:\n",
    "    retry_val = os.environ['RETRY_BACKOFF'].strip().split()[0]\n",
    "    try:\n",
    "        float(retry_val)\n",
    "        os.environ['RETRY_BACKOFF'] = retry_val\n",
    "    except ValueError:\n",
    "        os.environ['RETRY_BACKOFF'] = '1.0'\n",
    "\n",
    "def route(prompt: str, max_tokens: int = 200, temperature: float = 0.7):\n",
    "    \"\"\"Route prompt to appropriate model based on intent.\n",
    "    \n",
    "    Pipeline:\n",
    "    1. Detect intent using regex patterns\n",
    "    2. Select best model by capability + priority\n",
    "    3. Execute via Foundry Local SDK\n",
    "    \n",
    "    Args:\n",
    "        prompt: User input text\n",
    "        max_tokens: Maximum tokens in response\n",
    "        temperature: Sampling temperature (0-1)\n",
    "        \n",
    "    Returns:\n",
    "        Dict with: intent, model, output, tokens, usage, error\n",
    "    \"\"\"\n",
    "    intent = detect_intent(prompt)\n",
    "    model_alias = pick_model(intent)\n",
    "    \n",
    "    if not model_alias:\n",
    "        return {\n",
    "            'intent': intent,\n",
    "            'model': None,\n",
    "            'output': '',\n",
    "            'tokens': None,\n",
    "            'usage': {},\n",
    "            'error': 'No suitable model found'\n",
    "        }\n",
    "    \n",
    "    try:\n",
    "        # Call Foundry Local via workshop_utils\n",
    "        text, usage = chat_once(\n",
    "            model_alias,\n",
    "            messages=[{\"role\": \"user\", \"content\": prompt}],\n",
    "            max_tokens=max_tokens,\n",
    "            temperature=temperature\n",
    "        )\n",
    "        \n",
    "        # Extract token information\n",
    "        usage_info = {}\n",
    "        if usage:\n",
    "            usage_info['prompt_tokens'] = getattr(usage, 'prompt_tokens', None)\n",
    "            usage_info['completion_tokens'] = getattr(usage, 'completion_tokens', None)\n",
    "            usage_info['total_tokens'] = getattr(usage, 'total_tokens', None)\n",
    "        \n",
    "        # Estimate if not provided\n",
    "        if not usage_info.get('total_tokens'):\n",
    "            est_prompt = len(prompt) // 4\n",
    "            est_completion = len(text or '') // 4\n",
    "            usage_info['estimated_tokens'] = est_prompt + est_completion\n",
    "        \n",
    "        return {\n",
    "            'intent': intent,\n",
    "            'model': model_alias,\n",
    "            'output': (text or '').strip(),\n",
    "            'tokens': usage_info.get('total_tokens') or usage_info.get('estimated_tokens'),\n",
    "            'usage': usage_info,\n",
    "            'error': None\n",
    "        }\n",
    "    \n",
    "    except Exception as e:\n",
    "        return {\n",
    "            'intent': intent,\n",
    "            'model': model_alias,\n",
    "            'output': '',\n",
    "            'tokens': None,\n",
    "            'usage': {},\n",
    "            'error': f'{type(e).__name__}: {str(e)}'\n",
    "        }\n",
    "\n",
    "print('✅ Routing function ready')\n",
    "print('   Using Foundry Local SDK via workshop_utils')\n",
    "print('   Token tracking: Enabled')\n",
    "print('   Retry logic: Automatic with exponential backoff')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00a5c915",
   "metadata": {},
   "source": [
    "## 🎯 Стъпка 7: Провеждане на тестове за маршрутизация\n",
    "\n",
    "Тествайте цялостната система за маршрутизация с различни подсказки, за да демонстрирате:\n",
    "- Автоматично разпознаване на намерения\n",
    "- Интелигентен избор на модел\n",
    "- Маршрутизация с множество модели, като се запазват моделите\n",
    "- Проследяване на токени и показатели за производителност\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85c46ef4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "🎯 Running Intent-Based Routing Tests\n",
      "================================================================================\n",
      "\n",
      "[1/6] Testing prompt...\n",
      "Prompt: Refactor this Python function to make it more efficient and readable\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Expected Intent: code\n",
      "   Detected Intent: code ✅\n",
      "   Selected Model:  phi-3.5-mini\n",
      "   ✅ Response: To refactor a Python function for efficiency and readability, I would need to see the specific funct...\n",
      "   📊 Tokens: ~158 (estimated)\n",
      "\n",
      "[2/6] Testing prompt...\n",
      "Prompt: Summarize the key benefits of using small language models at the edge\n",
      "   Expected Intent: summarize\n",
      "   Detected Intent: summarize ✅\n",
      "   Selected Model:  phi-4-mini\n",
      "   ❌ Error: APIConnectionError: Connection error.\n",
      "\n",
      "[3/6] Testing prompt...\n",
      "Prompt: Classify this user feedback: The app is slow but the UI looks great\n",
      "   Expected Intent: classification\n",
      "   Detected Intent: classification ✅\n",
      "   Selected Model:  qwen2.5-0.5b\n",
      "   ❌ Error: APIConnectionError: Connection error.\n",
      "\n",
      "[4/6] Testing prompt...\n",
      "Prompt: Explain the difference between local and cloud inference\n",
      "   Expected Intent: general\n",
      "   Detected Intent: general ✅\n",
      "   Selected Model:  phi-4-mini\n",
      "   ❌ Error: APIConnectionError: Connection error.\n",
      "\n",
      "[5/6] Testing prompt...\n",
      "Prompt: Write a Python function to calculate the Fibonacci sequence\n"
     ]
    }
   ],
   "source": [
    "# Test prompts covering all intent categories\n",
    "test_cases = [\n",
    "    {\n",
    "        'prompt': 'Refactor this Python function to make it more efficient and readable',\n",
    "        'expected_intent': 'code'\n",
    "    },\n",
    "    {\n",
    "        'prompt': 'Summarize the key benefits of using small language models at the edge',\n",
    "        'expected_intent': 'summarize'\n",
    "    },\n",
    "    {\n",
    "        'prompt': 'Classify this user feedback: The app is slow but the UI looks great',\n",
    "        'expected_intent': 'classification'\n",
    "    },\n",
    "    {\n",
    "        'prompt': 'Explain the difference between local and cloud inference',\n",
    "        'expected_intent': 'general'\n",
    "    },\n",
    "    {\n",
    "        'prompt': 'Write a Python function to calculate the Fibonacci sequence',\n",
    "        'expected_intent': 'code'\n",
    "    },\n",
    "    {\n",
    "        'prompt': 'Give me a brief overview of the Phi model family',\n",
    "        'expected_intent': 'summarize'\n",
    "    }\n",
    "]\n",
    "\n",
    "print('🎯 Running Intent-Based Routing Tests')\n",
    "print('=' * 80)\n",
    "\n",
    "results = []\n",
    "for i, test in enumerate(test_cases, 1):\n",
    "    print(f'\\n[{i}/{len(test_cases)}] Testing prompt...')\n",
    "    print(f'Prompt: {test[\"prompt\"]}')\n",
    "    \n",
    "    result = route(test['prompt'], max_tokens=150)\n",
    "    results.append(result)\n",
    "    \n",
    "    print(f'   Expected Intent: {test[\"expected_intent\"]}')\n",
    "    print(f'   Detected Intent: {result[\"intent\"]} {\"✅\" if result[\"intent\"] == test[\"expected_intent\"] else \"⚠️\"}')\n",
    "    print(f'   Selected Model:  {result[\"model\"]}')\n",
    "    \n",
    "    if result['error']:\n",
    "        print(f'   ❌ Error: {result[\"error\"]}')\n",
    "    else:\n",
    "        output_preview = result['output'][:100] + '...' if len(result['output']) > 100 else result['output']\n",
    "        print(f'   ✅ Response: {output_preview}')\n",
    "        \n",
    "        tokens = result.get('tokens', 0)\n",
    "        if tokens:\n",
    "            usage = result.get('usage', {})\n",
    "            if 'estimated_tokens' in usage:\n",
    "                print(f'   📊 Tokens: ~{tokens} (estimated)')\n",
    "            else:\n",
    "                print(f'   📊 Tokens: {tokens}')\n",
    "\n",
    "# Summary statistics\n",
    "print('\\n' + '=' * 80)\n",
    "print('📊 ROUTING SUMMARY')\n",
    "print('=' * 80)\n",
    "\n",
    "success_count = sum(1 for r in results if not r['error'])\n",
    "total_tokens = sum(r.get('tokens', 0) or 0 for r in results if not r['error'])\n",
    "intent_accuracy = sum(1 for i, r in enumerate(results) if r['intent'] == test_cases[i]['expected_intent'])\n",
    "\n",
    "print(f'Total Prompts:        {len(results)}')\n",
    "print(f'✅ Successful:         {success_count}/{len(results)}')\n",
    "print(f'❌ Failed:             {len(results) - success_count}')\n",
    "print(f'🎯 Intent Accuracy:    {intent_accuracy}/{len(results)} ({intent_accuracy/len(results)*100:.1f}%)')\n",
    "print(f'📊 Total Tokens Used:  {total_tokens}')\n",
    "\n",
    "# Model usage distribution\n",
    "print('\\n📋 Model Usage Distribution:')\n",
    "model_counts = {}\n",
    "for r in results:\n",
    "    if r['model']:\n",
    "        model_counts[r['model']] = model_counts.get(r['model'], 0) + 1\n",
    "\n",
    "for model, count in sorted(model_counts.items(), key=lambda x: x[1], reverse=True):\n",
    "    percentage = (count / len(results)) * 100\n",
    "    print(f'   • {model}: {count} requests ({percentage:.1f}%)')\n",
    "\n",
    "if success_count == len(results):\n",
    "    print('\\n🎉 All routing tests passed successfully!')\n",
    "else:\n",
    "    print(f'\\n⚠️  {len(results) - success_count} test(s) failed')\n",
    "    print('   Check Foundry Local service: foundry service status')\n",
    "    print('   Verify models loaded: foundry model ls')\n",
    "\n",
    "print('=' * 80)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4764811e",
   "metadata": {},
   "source": [
    "## 🔧 Стъпка 8: Интерактивно тестване\n",
    "\n",
    "Изпробвайте свои собствени подсказки, за да видите системата за маршрутизиране в действие!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f8fdd51",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "🎯 Interactive Routing Test\n",
      "================================================================================\n",
      "Your prompt: Explain how model quantization reduces memory usage\n",
      "\n",
      "Detected Intent: general\n",
      "Selected Model:  phi-4-mini\n",
      "\n",
      "✅ Response:\n",
      "--------------------------------------------------------------------------------\n",
      "Model quantization is a technique used to reduce the memory footprint of a machine learning model, particularly deep learning models. It works by converting the high-precision weights of a neural network, typically represented as 32-bit floating-point numbers, into lower-precision representations, such as 8-bit integers or even binary values.\n",
      "\n",
      "\n",
      "The primary reason for quantization is to decrease the amount of memory required to store the model's parameters. Since floating-point numbers take up more space than integers, by quantizing the weights, we can significantly reduce the model's size. This reduction in size not only saves memory but also can lead to faster computation during inference, as integer operations are generally faster than floating-point operations on many hardware platforms.\n",
      "\n",
      "\n",
      "However, quantization can introduce some loss of accuracy because the lower precision representation may not capture the full range of values that the floating-point representation can. To mitigate this, techniques such as quantization-aware training can be used, where the model is trained with quantization in mind,\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      "📊 Tokens used: 292\n",
      "\n",
      "💡 Try different prompts to test routing behavior!\n",
      "Detected Intent: general\n",
      "Selected Model:  phi-4-mini\n",
      "\n",
      "✅ Response:\n",
      "--------------------------------------------------------------------------------\n",
      "Model quantization is a technique used to reduce the memory footprint of a machine learning model, particularly deep learning models. It works by converting the high-precision weights of a neural network, typically represented as 32-bit floating-point numbers, into lower-precision representations, such as 8-bit integers or even binary values.\n",
      "\n",
      "\n",
      "The primary reason for quantization is to decrease the amount of memory required to store the model's parameters. Since floating-point numbers take up more space than integers, by quantizing the weights, we can significantly reduce the model's size. This reduction in size not only saves memory but also can lead to faster computation during inference, as integer operations are generally faster than floating-point operations on many hardware platforms.\n",
      "\n",
      "\n",
      "However, quantization can introduce some loss of accuracy because the lower precision representation may not capture the full range of values that the floating-point representation can. To mitigate this, techniques such as quantization-aware training can be used, where the model is trained with quantization in mind,\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      "📊 Tokens used: 292\n",
      "\n",
      "💡 Try different prompts to test routing behavior!\n"
     ]
    }
   ],
   "source": [
    "# Interactive testing - modify the prompt and run this cell\n",
    "custom_prompt = \"Explain how model quantization reduces memory usage\"\n",
    "\n",
    "print('🎯 Interactive Routing Test')\n",
    "print('=' * 80)\n",
    "print(f'Your prompt: {custom_prompt}')\n",
    "print()\n",
    "\n",
    "result = route(custom_prompt, max_tokens=200)\n",
    "\n",
    "print(f'Detected Intent: {result[\"intent\"]}')\n",
    "print(f'Selected Model:  {result[\"model\"]}')\n",
    "print()\n",
    "\n",
    "if result['error']:\n",
    "    print(f'❌ Error: {result[\"error\"]}')\n",
    "else:\n",
    "    print('✅ Response:')\n",
    "    print('-' * 80)\n",
    "    print(result['output'])\n",
    "    print('-' * 80)\n",
    "    \n",
    "    if result['tokens']:\n",
    "        print(f'\\n📊 Tokens used: {result[\"tokens\"]}')\n",
    "\n",
    "print('\\n💡 Try different prompts to test routing behavior!')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c17226c",
   "metadata": {},
   "source": [
    "## 📊 Стъпка 9: Анализ на производителността\n",
    "\n",
    "Анализирайте производителността на системата за маршрутизация и използването на модела.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "805c688c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "⚡ Performance Benchmark\n",
      "================================================================================\n",
      "\n",
      "Prompt: Write a hello world function...\n",
      "   Model: phi-3.5-mini\n",
      "   Time: 3.31s\n",
      "   Tokens: 60\n",
      "\n",
      "Prompt: Write a hello world function...\n",
      "   Model: phi-3.5-mini\n",
      "   Time: 3.31s\n",
      "   Tokens: 60\n",
      "\n",
      "Prompt: Summarize: AI at the edge is powerful...\n",
      "   Model: phi-4-mini\n",
      "   Time: 49.67s\n",
      "   Tokens: 84\n",
      "\n",
      "Prompt: Summarize: AI at the edge is powerful...\n",
      "   Model: phi-4-mini\n",
      "   Time: 49.67s\n",
      "   Tokens: 84\n",
      "\n",
      "Prompt: Classify: Good product...\n",
      "   Model: qwen2.5-0.5b\n",
      "   Time: 7.21s\n",
      "   Tokens: 69\n",
      "\n",
      "Prompt: Classify: Good product...\n",
      "   Model: qwen2.5-0.5b\n",
      "   Time: 7.21s\n",
      "   Tokens: 69\n",
      "\n",
      "Prompt: Explain edge computing...\n",
      "   Model: phi-4-mini\n",
      "   Time: 49.67s\n",
      "   Tokens: 72\n",
      "\n",
      "================================================================================\n",
      "📊 Performance Statistics:\n",
      "   Average response time: 27.46s\n",
      "   Fastest response:      3.31s\n",
      "   Slowest response:      49.67s\n",
      "\n",
      "💡 Note: First request may be slower due to model initialization\n",
      "================================================================================\n",
      "\n",
      "Prompt: Explain edge computing...\n",
      "   Model: phi-4-mini\n",
      "   Time: 49.67s\n",
      "   Tokens: 72\n",
      "\n",
      "================================================================================\n",
      "📊 Performance Statistics:\n",
      "   Average response time: 27.46s\n",
      "   Fastest response:      3.31s\n",
      "   Slowest response:      49.67s\n",
      "\n",
      "💡 Note: First request may be slower due to model initialization\n",
      "================================================================================\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "# Performance benchmark\n",
    "benchmark_prompts = [\n",
    "    'Write a hello world function',\n",
    "    'Summarize: AI at the edge is powerful',\n",
    "    'Classify: Good product',\n",
    "    'Explain edge computing'\n",
    "]\n",
    "\n",
    "print('⚡ Performance Benchmark')\n",
    "print('=' * 80)\n",
    "\n",
    "timings = []\n",
    "for prompt in benchmark_prompts:\n",
    "    start = time.time()\n",
    "    result = route(prompt, max_tokens=50)\n",
    "    duration = time.time() - start\n",
    "    timings.append(duration)\n",
    "    \n",
    "    print(f'\\nPrompt: {prompt[:40]}...')\n",
    "    print(f'   Model: {result[\"model\"]}')\n",
    "    print(f'   Time: {duration:.2f}s')\n",
    "    if result.get('tokens'):\n",
    "        print(f'   Tokens: {result[\"tokens\"]}')\n",
    "\n",
    "print('\\n' + '=' * 80)\n",
    "print('📊 Performance Statistics:')\n",
    "print(f'   Average response time: {sum(timings)/len(timings):.2f}s')\n",
    "print(f'   Fastest response:      {min(timings):.2f}s')\n",
    "print(f'   Slowest response:      {max(timings):.2f}s')\n",
    "print('\\n💡 Note: First request may be slower due to model initialization')\n",
    "print('=' * 80)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7db64ff",
   "metadata": {},
   "source": [
    "## 🎓 Основни изводи и следващи стъпки\n",
    "\n",
    "### ✅ Какво научихте\n",
    "\n",
    "1. **Рутиране на база намерение**: Автоматично класифициране на заявки и насочване към специализирани модели  \n",
    "2. **Избор, съобразен с паметта**: Избиране на CPU модели според наличната системна RAM  \n",
    "3. **Задържане на множество модели**: Използвайте `--retain true`, за да запазите заредени няколко модела  \n",
    "4. **Производствени модели**: Логика за повторение, обработка на грешки и проследяване на токени  \n",
    "5. **Оптимизация за CPU**: Ефективно внедряване без изисквания за GPU  \n",
    "\n",
    "### 🚀 Идеи за експерименти\n",
    "\n",
    "1. **Добавяне на персонализирани намерения**:  \n",
    "   ```python\n",
    "   INTENT_RULES.append(\n",
    "       (re.compile(r'translate|convert', re.I), 'translation')\n",
    "   )\n",
    "   ```\n",
    "  \n",
    "2. **Зареждане на допълнителни модели**:  \n",
    "   ```bash\n",
    "   foundry model run llama-3.2-1b-cpu --retain true\n",
    "   ```\n",
    "  \n",
    "3. **Настройка на избора на модели**:  \n",
    "   - Регулирайте стойностите на приоритет в CATALOG  \n",
    "   - Добавете повече етикети за способности  \n",
    "   - Внедрете стратегии за резервни варианти  \n",
    "\n",
    "4. **Наблюдение на производителността**:  \n",
    "   ```python\n",
    "   import psutil\n",
    "   print(f\"Memory: {psutil.virtual_memory().percent}%\")\n",
    "   ```\n",
    "  \n",
    "\n",
    "### 📚 Допълнителни ресурси\n",
    "\n",
    "- **Foundry Local SDK**: https://github.com/microsoft/Foundry-Local  \n",
    "- **Примерни материали за работилници**: ../samples/  \n",
    "- **Курс за Edge AI**: ../../Module08/  \n",
    "\n",
    "### 💡 Най-добри практики\n",
    "\n",
    "✅ Използвайте CPU модели за последователно поведение на различни платформи  \n",
    "✅ Винаги проверявайте системната памет преди зареждане на множество модели  \n",
    "✅ Използвайте `--retain true` за сценарии с рутиране  \n",
    "✅ Внедрете правилна обработка на грешки и повторения  \n",
    "✅ Проследявайте използването на токени за оптимизация на разходите/производителността  \n",
    "\n",
    "---\n",
    "\n",
    "**🎉 Поздравления!** Създадохте готов за производство рутер на модели, базиран на намерения, използвайки Foundry Local SDK с модели, оптимизирани за CPU!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n\n<!-- CO-OP TRANSLATOR DISCLAIMER START -->\n**Отказ от отговорност**:  \nТози документ е преведен с помощта на AI услуга за превод [Co-op Translator](https://github.com/Azure/co-op-translator). Въпреки че се стремим към точност, моля, имайте предвид, че автоматичните преводи може да съдържат грешки или неточности. Оригиналният документ на неговия оригинален език трябва да се счита за авторитетен източник. За критична информация се препоръчва професионален човешки превод. Ние не носим отговорност за каквито и да било недоразумения или погрешни интерпретации, произтичащи от използването на този превод.\n<!-- CO-OP TRANSLATOR DISCLAIMER END -->\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "demo",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.15"
  },
  "coopTranslator": {
   "original_hash": "2fbaf6df2c479cfd0bb85547ef819dff",
   "translation_date": "2025-11-18T17:11:56+00:00",
   "source_file": "Workshop/notebooks/session06_models_router.ipynb",
   "language_code": "bg"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}