{ "cells": [ { "cell_type": "markdown", "id": "65ca1c8b", "metadata": {}, "source": [ "# الجلسة 1 – بدء تشغيل الدردشة (Foundry Local)\n", "\n", "يقوم هذا الدفتر ببدء تشغيل Foundry Local، وتنزيل الاسم المستعار للنموذج المفضل، ويقوم بإجراء إكمال دردشة قياسي وتدفق.\n" ] }, { "cell_type": "markdown", "id": "126def3c", "metadata": {}, "source": [ "# السيناريو\n", "تقدم هذه الجلسة الحد الأدنى المطلوب لجعل نموذج لغة صغير محلي يستجيب عبر Foundry Local. ستقوم بـ:\n", "- تثبيت SDK / تبعيات العميل.\n", "- تهيئة مدير Foundry Local لاسم مستعار مختار (الافتراضي: `phi-4-mini`).\n", "- تطبيق تصحيح دفاعي لتحمل الحقول الاختيارية في بيانات النموذج الوصفية.\n", "- إرسال طلب إكمال محادثة قياسي.\n", "- بث استجابة رمز‑ب‑رمز.\n", "\n", "الهدف هو التحقق من تشغيلك المحلي ومسار الشبكة قبل الانتقال إلى RAG أو التوجيه أو الوكلاء.\n" ] }, { "cell_type": "markdown", "id": "79dbb732", "metadata": {}, "source": [ "### شرح: تثبيت التبعيات\n", "تثبيت حزم Python المطلوبة لتشغيل هذا التدفق البسيط للدردشة:\n", "- `foundry-local-sdk`: إدارة النماذج المحلية ودورة حياة الخدمات.\n", "- `openai`: واجهة مألوفة لإكمال الدردشة.\n", "- `rich`: طباعة جميلة لتحسين وضوح المخرجات في الدفاتر.\n", "\n", "إعادة التشغيل آمنة (غير مكررة). يمكن تخطيها إذا كانت بيئتك تحتوي بالفعل على هذه الحزم.\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "09accd63", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.3\u001b[0m\n", "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "# Install required libraries (idempotent)\n", "%pip install -q foundry-local-sdk openai rich" ] }, { "cell_type": "markdown", "id": "1c32e699", "metadata": {}, "source": [ "### شرح: الواردات الأساسية\n", "يجلب الوحدات المستخدمة في جميع أنحاء الدفتر:\n", "- `FoundryLocalManager` للتفاعل مع بيئة تشغيل النموذج المحلي.\n", "- عميل `OpenAI` حتى نتمكن من إعادة استخدام واجهة برمجة التطبيقات المألوفة لإكمال الدردشة.\n", "- `rich.print` لإخراج منسق.\n", "\n", "لا تحدث أي مكالمات شبكية هنا—هذا فقط يجهز مساحة الأسماء.\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "73cb8c38", "metadata": {}, "outputs": [], "source": [ "import os\n", "from foundry_local import FoundryLocalManager\n", "from foundry_local.models import FoundryModelInfo\n", "from openai import OpenAI\n", "from rich import print" ] }, { "cell_type": "markdown", "id": "5fda4e9a", "metadata": {}, "source": [ "### شرح: تهيئة المدير وتصحيح البيانات الوصفية\n", "يقوم بتهيئة `FoundryLocalManager` للاسم المستعار المختار ويطبق تصحيحًا دفاعيًا لمعالجة استجابات الخدمة بشكل سلس حيث قد يكون `promptTemplate` قيمته `null`.\n", "\n", "النتائج الرئيسية:\n", "- تأكيد حالة الخدمة ونقطة النهاية.\n", "- عرض النماذج المخزنة مؤقتًا (التحقق من التخزين المحلي).\n", "- تحديد معرف النموذج المحدد للاسم المستعار (يُستخدم في مكالمات الدردشة اللاحقة).\n", "\n", "إذا واجهت مشاكل في التحقق من صحة البيانات الوصفية الخام للخدمة، يوضح هذا النمط كيفية التنقية دون الحاجة إلى تعديل SDK.\n" ] }, { "cell_type": "code", "execution_count": 3, "id": "2c0e087c", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Service running: True\n",
       "
\n" ], "text/plain": [ "\u001b[1;32mService running:\u001b[0m \u001b[3;92mTrue\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Endpoint: http://127.0.0.1:50262/v1\n",
       "
\n" ], "text/plain": [ "Endpoint: \u001b[4;94mhttp://127.0.0.1:50262/v1\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Cached models:\n",
       "[\n",
       "    FoundryModelInfo(\n",
       "        alias='phi-4-mini',\n",
       "        id='Phi-4-mini-instruct-generic-gpu:4',\n",
       "        version='4',\n",
       "        execution_provider='WebGpuExecutionProvider',\n",
       "        device_type=<DeviceType.GPU: 'GPU'>,\n",
       "        uri='azureml://registries/azureml/models/Phi-4-mini-instruct-generic-gpu/versions/4',\n",
       "        file_size_mb=3809,\n",
       "        prompt_template={\n",
       "            'system': '<|system|>{Content}<|end|>',\n",
       "            'user': '<|user|>{Content}<|end|>',\n",
       "            'assistant': '<|assistant|>{Content}<|end|>',\n",
       "            'prompt': '<|user|>{Content}<|end|><|assistant|>'\n",
       "        },\n",
       "        provider='AzureFoundry',\n",
       "        publisher='Microsoft',\n",
       "        license='MIT',\n",
       "        task='chat-completion',\n",
       "        ep_override=None\n",
       "    ),\n",
       "    FoundryModelInfo(\n",
       "        alias='qwen2.5-0.5b',\n",
       "        id='qwen2.5-0.5b-instruct-generic-gpu:3',\n",
       "        version='3',\n",
       "        execution_provider='WebGpuExecutionProvider',\n",
       "        device_type=<DeviceType.GPU: 'GPU'>,\n",
       "        uri='azureml://registries/azureml/models/qwen2.5-0.5b-instruct-generic-gpu/versions/3',\n",
       "        file_size_mb=700,\n",
       "        prompt_template={\n",
       "            'system': '<|im_start|>system\\n{Content}<|im_end|>',\n",
       "            'user': '<|im_start|>user\\n{Content}<|im_end|>',\n",
       "            'assistant': '<|im_start|>assistant\\n{Content}<|im_end|>',\n",
       "            'prompt': '<|im_start|>user\\n{Content}<|im_end|>\\n<|im_start|>assistant'\n",
       "        },\n",
       "        provider='AzureFoundry',\n",
       "        publisher='Microsoft',\n",
       "        license='apache-2.0',\n",
       "        task='chat-completion',\n",
       "        ep_override=None\n",
       "    ),\n",
       "    FoundryModelInfo(\n",
       "        alias='phi-3.5-mini',\n",
       "        id='Phi-3.5-mini-instruct-generic-gpu:1',\n",
       "        version='1',\n",
       "        execution_provider='WebGpuExecutionProvider',\n",
       "        device_type=<DeviceType.GPU: 'GPU'>,\n",
       "        uri='azureml://registries/azureml/models/Phi-3.5-mini-instruct-generic-gpu/versions/1',\n",
       "        file_size_mb=2211,\n",
       "        prompt_template={\n",
       "            'prompt': '<|user|>\\n{Content}<|end|>\\n<|assistant|>',\n",
       "            'assistant': '<|assistant|>\\n{Content}<|end|>'\n",
       "        },\n",
       "        provider='AzureFoundry',\n",
       "        publisher='Microsoft',\n",
       "        license='MIT',\n",
       "        task='chat-completion',\n",
       "        ep_override=None\n",
       "    )\n",
       "]\n",
       "
\n" ], "text/plain": [ "Cached models:\n", "\u001b[1m[\u001b[0m\n", " \u001b[1;35mFoundryModelInfo\u001b[0m\u001b[1m(\u001b[0m\n", " \u001b[33malias\u001b[0m=\u001b[32m'phi-4-mini'\u001b[0m,\n", " \u001b[33mid\u001b[0m=\u001b[32m'Phi-4-mini-instruct-generic-gpu:4'\u001b[0m,\n", " \u001b[33mversion\u001b[0m=\u001b[32m'4'\u001b[0m,\n", " \u001b[33mexecution_provider\u001b[0m=\u001b[32m'WebGpuExecutionProvider'\u001b[0m,\n", " \u001b[33mdevice_type\u001b[0m=\u001b[1m<\u001b[0m\u001b[1;95mDeviceType.GPU:\u001b[0m\u001b[39m \u001b[0m\u001b[32m'GPU'\u001b[0m\u001b[39m>,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33muri\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'azureml://registries/azureml/models/Phi-4-mini-instruct-generic-gpu/versions/4'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mfile_size_mb\u001b[0m\u001b[39m=\u001b[0m\u001b[1;36m3809\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mprompt_template\u001b[0m\u001b[39m=\u001b[0m\u001b[1;39m{\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'system'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|system|>\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|end|>'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'user'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|user|>\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|end|>'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'assistant'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|assistant|>\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|end|>'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'prompt'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|user|>\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|end|><|assistant|>'\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[1;39m}\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mprovider\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'AzureFoundry'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mpublisher\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'Microsoft'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mlicense\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'MIT'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mtask\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'chat-completion'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mep_override\u001b[0m\u001b[39m=\u001b[0m\u001b[3;35mNone\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[1;35mFoundryModelInfo\u001b[0m\u001b[1;39m(\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33malias\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'qwen2.5-0.5b'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mid\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'qwen2.5-0.5b-instruct-generic-gpu:3'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mversion\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'3'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mexecution_provider\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'WebGpuExecutionProvider'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mdevice_type\u001b[0m\u001b[39m=,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33muri\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'azureml://registries/azureml/models/qwen2.5-0.5b-instruct-generic-gpu/versions/3'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mfile_size_mb\u001b[0m\u001b[39m=\u001b[0m\u001b[1;36m700\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mprompt_template\u001b[0m\u001b[39m=\u001b[0m\u001b[1;39m{\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'system'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|im_start|>system\\n\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|im_end|>'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'user'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|im_start|>user\\n\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|im_end|>'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'assistant'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|im_start|>assistant\\n\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|im_end|>'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'prompt'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|im_start|>user\\n\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|im_end|>\\n<|im_start|>assistant'\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[1;39m}\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mprovider\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'AzureFoundry'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mpublisher\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'Microsoft'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mlicense\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'apache-2.0'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mtask\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'chat-completion'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mep_override\u001b[0m\u001b[39m=\u001b[0m\u001b[3;35mNone\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[1;39m)\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[1;35mFoundryModelInfo\u001b[0m\u001b[1;39m(\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33malias\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'phi-3.5-mini'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mid\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'Phi-3.5-mini-instruct-generic-gpu:1'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mversion\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'1'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mexecution_provider\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'WebGpuExecutionProvider'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mdevice_type\u001b[0m\u001b[39m=,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33muri\u001b[0m\u001b[39m=\u001b[0m\u001b[32m'azureml://registries/azureml/models/Phi-3.5-mini-instruct-generic-gpu/versions/1'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mfile_size_mb\u001b[0m\u001b[39m=\u001b[0m\u001b[1;36m2211\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[33mprompt_template\u001b[0m\u001b[39m=\u001b[0m\u001b[1;39m{\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'prompt'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|user|>\\n\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|end|>\\n<|assistant|>'\u001b[0m\u001b[39m,\u001b[0m\n", "\u001b[39m \u001b[0m\u001b[32m'assistant'\u001b[0m\u001b[39m: \u001b[0m\u001b[32m'<|assistant|>\\n\u001b[0m\u001b[32m{\u001b[0m\u001b[32mContent\u001b[0m\u001b[32m}\u001b[0m\u001b[32m<|end|\u001b[0m\u001b[32m>\u001b[0m\u001b[32m'\u001b[0m\n", " \u001b[1m}\u001b[0m,\n", " \u001b[33mprovider\u001b[0m=\u001b[32m'AzureFoundry'\u001b[0m,\n", " \u001b[33mpublisher\u001b[0m=\u001b[32m'Microsoft'\u001b[0m,\n", " \u001b[33mlicense\u001b[0m=\u001b[32m'MIT'\u001b[0m,\n", " \u001b[33mtask\u001b[0m=\u001b[32m'chat-completion'\u001b[0m,\n", " \u001b[33mep_override\u001b[0m=\u001b[3;35mNone\u001b[0m\n", " \u001b[1m)\u001b[0m\n", "\u001b[1m]\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
Using model id: Phi-4-mini-instruct-generic-gpu:4\n",
       "
\n" ], "text/plain": [ "Using model id: Phi-\u001b[1;36m4\u001b[0m-mini-instruct-generic-gpu:\u001b[1;36m4\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Monkeypatch to tolerate service responses where promptTemplate is null\n", "_original_from_list_response = FoundryModelInfo.from_list_response\n", "\n", "def _safe_from_list_response(response): # type: ignore\n", " try:\n", " if isinstance(response, dict) and response.get(\"promptTemplate\") is None:\n", " # Normalize to empty dict so pydantic validation passes\n", " response[\"promptTemplate\"] = {}\n", " except Exception as e: # pragma: no cover\n", " print(f\"[yellow]Warning: safe wrapper encountered issue normalizing promptTemplate: {e}[/yellow]\")\n", " return _original_from_list_response(response)\n", "\n", "# Apply patch only once\n", "if getattr(FoundryModelInfo.from_list_response, \"__name__\", \"\") != \"_safe_from_list_response\":\n", " FoundryModelInfo.from_list_response = staticmethod(_safe_from_list_response) # type: ignore\n", "\n", "ALIAS = os.getenv('FOUNDRY_LOCAL_ALIAS', 'phi-4-mini')\n", "manager = FoundryLocalManager(ALIAS)\n", "print(f'[bold green]Service running:[/bold green] {manager.is_service_running()}')\n", "print(f'Endpoint: {manager.endpoint}')\n", "print('Cached models:', manager.list_cached_models())\n", "model_id = manager.get_model_info(ALIAS).id\n", "print(f'Using model id: {model_id}')" ] }, { "cell_type": "markdown", "id": "3fbb2059", "metadata": {}, "source": [ "### شرح: إكمال الدردشة الأساسي\n", "يقوم بإنشاء عميل متوافق مع `OpenAI` يشير إلى نقطة النهاية المحلية لـ Foundry ويقوم بتنفيذ عملية إكمال دردشة واحدة غير متدفقة. التركيز هنا:\n", "- التأكد من أن النموذج يستجيب بدون أخطاء.\n", "- التحقق من زمن الاستجابة / تنسيق الإخراج.\n", "- الحفاظ على قيمة `max_tokens` منخفضة لتوفير الموارد.\n", "\n", "إذا فشلت هذه العملية، تحقق مرة أخرى من تشغيل خدمة Foundry Local بشكل صحيح ومن أن الاسم المستعار يتم حله بشكل صحيح.\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "28fb04dd", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Local inference for privacy refers to the practice of performing data analysis on a local device without sending \n",
       "sensitive information to a central server. Two benefits of this approach are:\n",
       "\n",
       "\n",
       "1. **Enhanced Privacy**: Local inference keeps personal data on the user's device, reducing the risk of data \n",
       "breaches and unauthorized access. Since the data is not transmitted over the network, it is less susceptible to \n",
       "interception by malicious actors.\n",
       "\n",
       "\n",
       "2. **Data Sovereignty**: Users retain control over their data, as it does not leave their device. This means that \n",
       "individuals or organizations can comply with local data protection regulations, such as the General\n",
       "
\n" ], "text/plain": [ "Local inference for privacy refers to the practice of performing data analysis on a local device without sending \n", "sensitive information to a central server. Two benefits of this approach are:\n", "\n", "\n", "\u001b[1;36m1\u001b[0m. **Enhanced Privacy**: Local inference keeps personal data on the user's device, reducing the risk of data \n", "breaches and unauthorized access. Since the data is not transmitted over the network, it is less susceptible to \n", "interception by malicious actors.\n", "\n", "\n", "\u001b[1;36m2\u001b[0m. **Data Sovereignty**: Users retain control over their data, as it does not leave their device. This means that \n", "individuals or organizations can comply with local data protection regulations, such as the General\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "client = OpenAI(base_url=manager.endpoint, api_key=manager.api_key or 'not-needed')\n", "prompt = 'List two benefits of local inference for privacy.'\n", "resp = client.chat.completions.create(\n", " model=model_id,\n", " messages=[{'role':'user','content':prompt}],\n", " max_tokens=120,\n", " temperature=0.5\n", ")\n", "print(resp.choices[0].message.content)" ] }, { "cell_type": "markdown", "id": "f2091aa0", "metadata": {}, "source": [ "### شرح: إكمال الدردشة المتدفق\n", "يوضح كيفية تدفق الرموز لتحسين زمن الاستجابة المدرك وتجربة المستخدم التفاعلية. يقوم الحلقة بطباعة التغييرات التدريجية فور وصولها:\n", "- مفيد لواجهات الدردشة حيث يكون للإخراج الجزئي المبكر أهمية.\n", "- يتيح لك قياس معدل تدفق الرموز مقابل زمن إكمال النص بالكامل.\n", "\n", "يمكنك تعديل هذا النمط لتجميع الرموز، تحديث عنصر واجهة التقدم، أو إيقاف العملية أثناء التوليد.\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "5e85a2e8", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
Edge
\n" ], "text/plain": [ "Edge" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 AI
\n" ], "text/plain": [ " AI" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 refers
\n" ], "text/plain": [ " refers" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 to
\n" ], "text/plain": [ " to" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 artificial
\n" ], "text/plain": [ " artificial" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 intelligence
\n" ], "text/plain": [ " intelligence" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 algorithms
\n" ], "text/plain": [ " algorithms" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 and
\n" ], "text/plain": [ " and" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 models
\n" ], "text/plain": [ " models" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 that
\n" ], "text/plain": [ " that" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 are
\n" ], "text/plain": [ " are" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 deployed
\n" ], "text/plain": [ " deployed" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 at
\n" ], "text/plain": [ " at" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 the
\n" ], "text/plain": [ " the" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 edge
\n" ], "text/plain": [ " edge" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 of
\n" ], "text/plain": [ " of" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 the
\n" ], "text/plain": [ " the" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 network
\n" ], "text/plain": [ " network" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
,
\n" ], "text/plain": [ "," ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 closer
\n" ], "text/plain": [ " closer" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 to
\n" ], "text/plain": [ " to" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 the
\n" ], "text/plain": [ " the" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 source
\n" ], "text/plain": [ " source" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 of
\n" ], "text/plain": [ " of" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 data
\n" ], "text/plain": [ " data" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
,
\n" ], "text/plain": [ "," ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 to
\n" ], "text/plain": [ " to" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 enable
\n" ], "text/plain": [ " enable" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 real
\n" ], "text/plain": [ " real" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
-time
\n" ], "text/plain": [ "-time" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 processing
\n" ], "text/plain": [ " processing" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 and
\n" ], "text/plain": [ " and" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 decision
\n" ], "text/plain": [ " decision" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
-making
\n" ], "text/plain": [ "-making" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 with
\n" ], "text/plain": [ " with" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 reduced
\n" ], "text/plain": [ " reduced" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 latency
\n" ], "text/plain": [ " latency" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 and
\n" ], "text/plain": [ " and" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 bandwidth
\n" ], "text/plain": [ " bandwidth" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
 usage
\n" ], "text/plain": [ " usage" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
.
\n" ], "text/plain": [ "." ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n",
       "
\n" ], "text/plain": [ "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Streaming example\n", "stream = client.chat.completions.create(\n", " model=model_id,\n", " messages=[{'role':'user','content':'Give a one-sentence definition of edge AI.'}],\n", " stream=True,\n", " max_tokens=60,\n", " temperature=0.4\n", ")\n", "for chunk in stream:\n", " delta = chunk.choices[0].delta\n", " if delta and delta.content:\n", " print(delta.content, end='', flush=True)\n", "print()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n**إخلاء المسؤولية**: \nتم ترجمة هذا المستند باستخدام خدمة الترجمة بالذكاء الاصطناعي [Co-op Translator](https://github.com/Azure/co-op-translator). بينما نسعى لتحقيق الدقة، يرجى العلم أن الترجمات الآلية قد تحتوي على أخطاء أو عدم دقة. يجب اعتبار المستند الأصلي بلغته الأصلية المصدر الرسمي. للحصول على معلومات حاسمة، يُوصى بالترجمة البشرية الاحترافية. نحن غير مسؤولين عن أي سوء فهم أو تفسيرات خاطئة تنشأ عن استخدام هذه الترجمة.\n\n" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.12" }, "coopTranslator": { "original_hash": "3c07eacf0aa63b8dadf6733414138d47", "translation_date": "2025-11-11T20:58:42+00:00", "source_file": "Workshop/notebooks/session01_chat_bootstrap.ipynb", "language_code": "ar" } }, "nbformat": 4, "nbformat_minor": 5 }