{ "cells": [ { "cell_type": "markdown", "id": "0382ebeb", "metadata": {}, "source": [ "# Sample 04: Chainlit Web Application\n", "\n", "This notebook demonstrates how to create and understand a modern Chainlit web application for conversational AI using Microsoft Foundry Local with the OpenAI SDK.\n", "\n", "## Overview\n", "\n", "Chainlit is a powerful framework for building conversational AI applications with modern web interfaces. This sample shows:\n", "\n", "- šŸŒ **Modern Web UI**: Professional chat interface with real-time updates\n", "- šŸ”„ **Streaming Responses**: Real-time message streaming for better UX\n", "- šŸŽÆ **OpenAI SDK Integration**: Proper API client setup with Foundry Local\n", "- šŸ›”ļø **Error Handling**: Graceful fallbacks and user-friendly error messages\n", "- āš™ļø **Production Ready**: Enterprise-grade configuration and deployment patterns" ] }, { "cell_type": "markdown", "id": "8c9b131d", "metadata": {}, "source": [ "## Prerequisites and Setup\n", "\n", "Before running this notebook, ensure you have the required packages:" ] }, { "cell_type": "code", "execution_count": 14, "id": "b547d010", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com\n", "Requirement already satisfied: chainlit in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (2.8.0)\n", "Requirement already satisfied: openai in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (1.109.0)\n", "Requirement already satisfied: foundry-local-sdk in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (0.5.0)\n", "Requirement already satisfied: aiofiles<25.0.0,>=23.1.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (24.1.0)\n", "Requirement already satisfied: asyncer<0.1.0,>=0.0.8 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.0.8)\n", "Requirement already satisfied: click<9.0.0,>=8.1.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (8.3.0)\n", "Requirement already satisfied: dataclasses-json<0.7.0,>=0.6.7 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.6.7)\n", "Requirement already satisfied: fastapi<0.117,>=0.116.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.116.2)\n", "Requirement already satisfied: filetype<2.0.0,>=1.2.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (1.2.0)\n", "Requirement already satisfied: httpx>=0.23.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.27.2)\n", "Requirement already satisfied: lazify<0.5.0,>=0.4.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.4.0)\n", "Requirement already satisfied: literalai==0.1.201 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.1.201)\n", "Requirement already satisfied: mcp<2.0.0,>=1.11.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (1.14.1)\n", "Requirement already satisfied: nest-asyncio<2.0.0,>=1.6.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (1.6.0)\n", "Requirement already satisfied: packaging>=23.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (24.1)\n", "Requirement already satisfied: pydantic-settings>=2.10.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (2.10.1)\n", "Requirement already satisfied: pydantic<3,>=2.7.2 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (2.11.9)\n", "Requirement already satisfied: pyjwt<3.0.0,>=2.8.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (2.10.1)\n", "Requirement already satisfied: python-dotenv<2.0.0,>=1.0.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (1.1.1)\n", "Requirement already satisfied: python-multipart<1.0.0,>=0.0.18 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.0.20)\n", "Requirement already satisfied: python-socketio<6.0.0,>=5.11.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (5.13.0)\n", "Requirement already satisfied: starlette>=0.47.2 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.48.0)\n", "Requirement already satisfied: syncer<3.0.0,>=2.0.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (2.0.3)\n", "Requirement already satisfied: tomli<3.0.0,>=2.0.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (2.2.1)\n", "Requirement already satisfied: uvicorn>=0.35.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.37.0)\n", "Requirement already satisfied: watchfiles<1.0.0,>=0.20.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from chainlit) (0.24.0)\n", "Requirement already satisfied: chevron>=0.14.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from literalai==0.1.201->chainlit) (0.14.0)\n", "Requirement already satisfied: traceloop-sdk>=0.33.12 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: anyio<5,>=3.5.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from openai) (4.11.0)\n", "Requirement already satisfied: distro<2,>=1.7.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from openai) (1.9.0)\n", "Requirement already satisfied: jiter<1,>=0.4.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from openai) (0.11.0)\n", "Requirement already satisfied: sniffio in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from openai) (1.3.1)\n", "Requirement already satisfied: tqdm>4 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from openai) (4.66.5)\n", "Requirement already satisfied: typing-extensions<5,>=4.11 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from openai) (4.12.2)\n", "Requirement already satisfied: idna>=2.8 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from anyio<5,>=3.5.0->openai) (3.10)\n", "Requirement already satisfied: colorama in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from click<9.0.0,>=8.1.3->chainlit) (0.4.6)\n", "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from dataclasses-json<0.7.0,>=0.6.7->chainlit) (3.26.1)\n", "Requirement already satisfied: typing-inspect<1,>=0.4.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from dataclasses-json<0.7.0,>=0.6.7->chainlit) (0.9.0)\n", "Requirement already satisfied: certifi in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from httpx>=0.23.0->chainlit) (2024.8.30)\n", "Requirement already satisfied: httpcore==1.* in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from httpx>=0.23.0->chainlit) (1.0.5)\n", "Requirement already satisfied: h11<0.15,>=0.13 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from httpcore==1.*->httpx>=0.23.0->chainlit) (0.14.0)\n", "Requirement already satisfied: httpx-sse>=0.4 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from mcp<2.0.0,>=1.11.0->chainlit) (0.4.1)\n", "Requirement already satisfied: jsonschema>=4.20.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from mcp<2.0.0,>=1.11.0->chainlit) (4.23.0)\n", "Requirement already satisfied: pywin32>=310 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from mcp<2.0.0,>=1.11.0->chainlit) (311)\n", "Requirement already satisfied: sse-starlette>=1.6.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from mcp<2.0.0,>=1.11.0->chainlit) (3.0.2)\n", "Requirement already satisfied: annotated-types>=0.6.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from pydantic<3,>=2.7.2->chainlit) (0.7.0)\n", "Requirement already satisfied: pydantic-core==2.33.2 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from pydantic<3,>=2.7.2->chainlit) (2.33.2)\n", "Requirement already satisfied: typing-inspection>=0.4.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from pydantic<3,>=2.7.2->chainlit) (0.4.1)\n", "Requirement already satisfied: bidict>=0.21.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from python-socketio<6.0.0,>=5.11.0->chainlit) (0.23.1)\n", "Requirement already satisfied: python-engineio>=4.11.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from python-socketio<6.0.0,>=5.11.0->chainlit) (4.12.2)\n", "Requirement already satisfied: attrs>=22.2.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from jsonschema>=4.20.0->mcp<2.0.0,>=1.11.0->chainlit) (24.2.0)\n", "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from jsonschema>=4.20.0->mcp<2.0.0,>=1.11.0->chainlit) (2023.12.1)\n", "Requirement already satisfied: referencing>=0.28.4 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from jsonschema>=4.20.0->mcp<2.0.0,>=1.11.0->chainlit) (0.35.1)\n", "Requirement already satisfied: rpds-py>=0.7.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from jsonschema>=4.20.0->mcp<2.0.0,>=1.11.0->chainlit) (0.20.0)\n", "Requirement already satisfied: simple-websocket>=0.10.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from python-engineio>=4.11.0->python-socketio<6.0.0,>=5.11.0->chainlit) (1.1.0)\n", "Requirement already satisfied: aiohttp<4.0.0,>=3.11.11 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (3.12.15)\n", "Requirement already satisfied: cuid<0.5,>=0.4 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.4)\n", "Requirement already satisfied: deprecated<2.0.0,>=1.2.14 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.2.18)\n", "Requirement already satisfied: jinja2<4.0.0,>=3.1.5 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (3.1.6)\n", "Requirement already satisfied: opentelemetry-api<2.0.0,>=1.28.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.37.0)\n", "Requirement already satisfied: opentelemetry-exporter-otlp-proto-grpc<2.0.0,>=1.28.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.37.0)\n", "Requirement already satisfied: opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.28.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.37.0)\n", "Requirement already satisfied: opentelemetry-instrumentation-alephalpha==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-anthropic==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-bedrock==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-chromadb==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-cohere==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-crewai==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-google-generativeai==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-groq==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-haystack==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-lancedb==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-langchain==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-llamaindex==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-logging>=0.57b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: opentelemetry-instrumentation-marqo==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-mcp==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-milvus==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-mistralai==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-ollama==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-openai==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-openai-agents==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-pinecone==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-qdrant==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-redis>=0.50b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: opentelemetry-instrumentation-replicate==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-requests>=0.50b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: opentelemetry-instrumentation-sagemaker==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-sqlalchemy>=0.50b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: opentelemetry-instrumentation-threading>=0.50b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: opentelemetry-instrumentation-together==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-transformers==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-urllib3>=0.50b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: opentelemetry-instrumentation-vertexai==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-watsonx==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-weaviate==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-instrumentation-writer==0.47.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.47.3)\n", "Requirement already satisfied: opentelemetry-sdk<2.0.0,>=1.28.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.37.0)\n", "Requirement already satisfied: opentelemetry-semantic-conventions-ai<0.5.0,>=0.4.13 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.4.13)\n", "Requirement already satisfied: posthog<4,>3.0.2 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (3.25.0)\n", "Requirement already satisfied: tenacity<10.0,>=8.2.3 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (9.1.2)\n", "Requirement already satisfied: opentelemetry-instrumentation>=0.50b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-instrumentation-alephalpha==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: opentelemetry-semantic-conventions>=0.50b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-instrumentation-alephalpha==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: anthropic>=0.17.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-instrumentation-bedrock==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.68.0)\n", "Requirement already satisfied: tokenizers>=0.13.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-instrumentation-bedrock==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.20.0)\n", "Requirement already satisfied: inflection<0.6.0,>=0.5.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-instrumentation-llamaindex==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.5.1)\n", "Requirement already satisfied: opentelemetry-exporter-otlp<2.0.0,>=1.34.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-instrumentation-mcp==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.37.0)\n", "Requirement already satisfied: mypy-extensions>=0.3.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7.0,>=0.6.7->chainlit) (1.1.0)\n", "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from aiohttp<4.0.0,>=3.11.11->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (2.6.1)\n", "Requirement already satisfied: aiosignal>=1.4.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from aiohttp<4.0.0,>=3.11.11->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.4.0)\n", "Requirement already satisfied: frozenlist>=1.1.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from aiohttp<4.0.0,>=3.11.11->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.4.1)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from aiohttp<4.0.0,>=3.11.11->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (6.1.0)\n", "Requirement already satisfied: propcache>=0.2.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from aiohttp<4.0.0,>=3.11.11->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.3.2)\n", "Requirement already satisfied: yarl<2.0,>=1.17.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from aiohttp<4.0.0,>=3.11.11->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.20.1)\n", "Requirement already satisfied: wrapt<2,>=1.10 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from deprecated<2.0.0,>=1.2.14->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.17.3)\n", "Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from jinja2<4.0.0,>=3.1.5->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (2.1.5)\n", "Requirement already satisfied: importlib-metadata<8.8.0,>=6.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-api<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (8.5.0)\n", "Requirement already satisfied: googleapis-common-protos~=1.57 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-exporter-otlp-proto-grpc<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.70.0)\n", "Requirement already satisfied: grpcio<2.0.0,>=1.63.2 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-exporter-otlp-proto-grpc<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.75.0)\n", "Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.37.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-exporter-otlp-proto-grpc<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.37.0)\n", "Requirement already satisfied: opentelemetry-proto==1.37.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-exporter-otlp-proto-grpc<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.37.0)\n", "Requirement already satisfied: protobuf<7.0,>=5.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-proto==1.37.0->opentelemetry-exporter-otlp-proto-grpc<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (6.32.1)\n", "Requirement already satisfied: requests~=2.7 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (2.32.3)\n", "Requirement already satisfied: opentelemetry-util-http==0.58b0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from opentelemetry-instrumentation-requests>=0.50b0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.58b0)\n", "Requirement already satisfied: six>=1.5 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from posthog<4,>3.0.2->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.16.0)\n", "Requirement already satisfied: monotonic>=1.5 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from posthog<4,>3.0.2->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (1.6)\n", "Requirement already satisfied: backoff>=1.10.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from posthog<4,>3.0.2->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (2.2.1)\n", "Requirement already satisfied: python-dateutil>2.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from posthog<4,>3.0.2->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (2.9.0)\n", "Requirement already satisfied: wsproto in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from simple-websocket>=0.10.0->python-engineio>=4.11.0->python-socketio<6.0.0,>=5.11.0->chainlit) (1.2.0)\n", "Requirement already satisfied: docstring-parser<1,>=0.15 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from anthropic>=0.17.0->opentelemetry-instrumentation-bedrock==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.16)\n", "Requirement already satisfied: zipp>=3.20 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from importlib-metadata<8.8.0,>=6.0->opentelemetry-api<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (3.20.2)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from requests~=2.7->opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (3.3.2)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from requests~=2.7->opentelemetry-exporter-otlp-proto-http<2.0.0,>=1.28.0->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (2.2.3)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from tokenizers>=0.13.0->opentelemetry-instrumentation-bedrock==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (0.25.1)\n", "Requirement already satisfied: filelock in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->opentelemetry-instrumentation-bedrock==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (3.16.1)\n", "Requirement already satisfied: fsspec>=2023.5.0 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->opentelemetry-instrumentation-bedrock==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (2024.6.1)\n", "Requirement already satisfied: pyyaml>=5.1 in c:\\users\\leestott\\appdata\\local\\miniforge\\envs\\pydev\\lib\\site-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->opentelemetry-instrumentation-bedrock==0.47.3->traceloop-sdk>=0.33.12->literalai==0.1.201->chainlit) (6.0.2)\n" ] } ], "source": [ "# Install required packages\n", "!pip install chainlit openai foundry-local-sdk" ] }, { "cell_type": "markdown", "id": "dc08221b", "metadata": {}, "source": [ "## Understanding the Chainlit Application\n", "\n", "Let's examine the structure and key components of our Chainlit application:" ] }, { "cell_type": "code", "execution_count": 15, "id": "6569e528", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "āœ… Foundry Local SDK is available\n", "šŸ“¦ Libraries imported successfully\n" ] } ], "source": [ "# Import necessary libraries\n", "import chainlit as cl\n", "import os\n", "import sys\n", "from typing import Optional, Dict, Any\n", "from openai import OpenAI, AsyncOpenAI\n", "import asyncio\n", "\n", "# Check for Foundry Local SDK\n", "try:\n", " from foundry_local import FoundryLocalManager\n", " FOUNDRY_SDK_AVAILABLE = True\n", " print(\"āœ… Foundry Local SDK is available\")\n", "except ImportError:\n", " FOUNDRY_SDK_AVAILABLE = False\n", " print(\"āš ļø Foundry Local SDK not available, will use manual configuration\")\n", "\n", "print(\"šŸ“¦ Libraries imported successfully\")" ] }, { "cell_type": "markdown", "id": "055d20de", "metadata": {}, "source": [ "## Client Configuration Class\n", "\n", "This class handles the OpenAI client setup with Foundry Local integration:" ] }, { "cell_type": "code", "execution_count": 16, "id": "638523d5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸ”„ Initializing Foundry Local SDK with model: phi-4-mini...\n", "2025-09-23 21:43:21 - HTTP Request: GET http://127.0.0.1:51211/foundry/list \"HTTP/1.1 200 OK\"\n", "2025-09-23 21:43:21 - HTTP Request: GET http://127.0.0.1:51211/openai/models \"HTTP/1.1 200 OK\"\n", "2025-09-23 21:43:21 - HTTP Request: GET http://127.0.0.1:51211/openai/load/Phi-4-mini-instruct-cuda-gpu?ttl=600&ep= \"HTTP/1.1 200 OK\"\n", "āœ… Foundry Local SDK initialized at http://127.0.0.1:51211/v1\n", "\n", "šŸ“Š **Client Initialization Result:**\n", " Status: success\n", " Method: foundry_sdk\n", " Base_Url: http://127.0.0.1:51211/v1\n", " Model: phi-4-mini\n" ] } ], "source": [ "class FoundryClientManager:\n", " \"\"\"Manages OpenAI client setup for Foundry Local integration.\"\"\"\n", " \n", " def __init__(self, model_name: str = \"phi-4-mini\"):\n", " self.model_name = model_name\n", " self.client = None\n", " self.async_client = None\n", " self.base_url = None\n", " self.api_key = None\n", " \n", " def _get_fallback_config(self) -> tuple[str, str]:\n", " \"\"\"Get fallback configuration from environment variables.\"\"\"\n", " base_url = os.getenv(\"BASE_URL\", \"http://localhost:8000\")\n", " api_key = os.getenv(\"API_KEY\", \"\")\n", " return base_url, api_key\n", " \n", " def initialize_clients(self) -> Dict[str, Any]:\n", " \"\"\"Initialize both sync and async OpenAI clients.\"\"\"\n", " if FOUNDRY_SDK_AVAILABLE:\n", " try:\n", " print(f\"šŸ”„ Initializing Foundry Local SDK with model: {self.model_name}...\")\n", " manager = FoundryLocalManager(self.model_name)\n", " \n", " self.base_url = manager.endpoint\n", " self.api_key = manager.api_key\n", " \n", " # Create both sync and async clients\n", " self.client = OpenAI(\n", " base_url=self.base_url,\n", " api_key=self.api_key\n", " )\n", " \n", " self.async_client = AsyncOpenAI(\n", " base_url=self.base_url,\n", " api_key=self.api_key\n", " )\n", " \n", " print(f\"āœ… Foundry Local SDK initialized at {self.base_url}\")\n", " return {\n", " \"status\": \"success\",\n", " \"method\": \"foundry_sdk\",\n", " \"base_url\": self.base_url,\n", " \"model\": self.model_name\n", " }\n", " \n", " except Exception as e:\n", " print(f\"āš ļø Foundry SDK failed ({e}), falling back to manual configuration\")\n", " \n", " # Fallback to manual configuration\n", " self.base_url, self.api_key = self._get_fallback_config()\n", " \n", " self.client = OpenAI(\n", " base_url=f\"{self.base_url}/v1\",\n", " api_key=self.api_key\n", " )\n", " \n", " self.async_client = AsyncOpenAI(\n", " base_url=f\"{self.base_url}/v1\",\n", " api_key=self.api_key\n", " )\n", " \n", " print(f\"šŸ”§ Manual configuration at {self.base_url}/v1\")\n", " return {\n", " \"status\": \"success\",\n", " \"method\": \"manual\",\n", " \"base_url\": f\"{self.base_url}/v1\",\n", " \"model\": self.model_name\n", " }\n", " \n", " async def test_connection(self) -> Dict[str, Any]:\n", " \"\"\"Test the connection to Foundry Local service.\"\"\"\n", " try:\n", " # Try to list available models\n", " models = await self.async_client.models.list()\n", " available_models = [model.id for model in models.data]\n", " \n", " # Test with a simple completion\n", " response = await self.async_client.chat.completions.create(\n", " model=self.model_name,\n", " messages=[{\"role\": \"user\", \"content\": \"Hello, are you working?\"}],\n", " max_tokens=50\n", " )\n", " \n", " return {\n", " \"status\": \"healthy\",\n", " \"available_models\": available_models,\n", " \"current_model\": self.model_name,\n", " \"test_response\": response.choices[0].message.content,\n", " \"base_url\": self.base_url\n", " }\n", " \n", " except Exception as e:\n", " return {\n", " \"status\": \"error\",\n", " \"error\": str(e),\n", " \"base_url\": self.base_url\n", " }\n", "\n", "# Create a client manager instance\n", "client_manager = FoundryClientManager(\"phi-4-mini\")\n", "init_result = client_manager.initialize_clients()\n", "\n", "print(f\"\\nšŸ“Š **Client Initialization Result:**\")\n", "for key, value in init_result.items():\n", " print(f\" {key.title()}: {value}\")" ] }, { "cell_type": "markdown", "id": "05013074", "metadata": {}, "source": [ "## Connection Testing\n", "\n", "Let's test our connection to the Foundry Local service:" ] }, { "cell_type": "code", "execution_count": 17, "id": "96614f4e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸ” **Testing Foundry Local Connection**\n", "==================================================\n", "2025-09-23 21:43:24 - HTTP Request: GET http://127.0.0.1:51211/v1/models \"HTTP/1.1 200 OK\"\n", "2025-09-23 21:43:24 - HTTP Request: POST http://127.0.0.1:51211/v1/chat/completions \"HTTP/1.1 400 Bad Request\"\n", "2025-09-23 21:43:24 - HTTP Request: POST http://127.0.0.1:51211/v1/chat/completions \"HTTP/1.1 400 Bad Request\"\n", "āŒ **Connection Status:** Error\n", "šŸ”— **Base URL:** http://127.0.0.1:51211/v1\n", "āš ļø **Error:** Error code: 400\n", "\n", "šŸ”§ **Troubleshooting:**\n", "1. Check if Foundry Local is running: foundry service status\n", "2. Start a model: foundry model run phi-4-mini\n", "3. Verify the endpoint URL and port\n" ] } ], "source": [ "# Test the connection asynchronously\n", "async def test_service_connection():\n", " \"\"\"Test connection to Foundry Local service.\"\"\"\n", " print(\"šŸ” **Testing Foundry Local Connection**\")\n", " print(\"=\" * 50)\n", " \n", " health_check = await client_manager.test_connection()\n", " \n", " if health_check[\"status\"] == \"healthy\":\n", " print(\"āœ… **Connection Status:** Healthy\")\n", " print(f\"šŸ”— **Base URL:** {health_check['base_url']}\")\n", " print(f\"šŸ¤– **Current Model:** {health_check['current_model']}\")\n", " print(f\"šŸ’¬ **Test Response:** {health_check['test_response']}\")\n", " \n", " print(f\"\\nšŸ“‹ **Available Models ({len(health_check['available_models'])}):**\")\n", " for i, model in enumerate(health_check['available_models'], 1):\n", " current = \" (current)\" if model == health_check['current_model'] else \"\"\n", " print(f\" {i}. {model}{current}\")\n", " else:\n", " print(\"āŒ **Connection Status:** Error\")\n", " print(f\"šŸ”— **Base URL:** {health_check['base_url']}\")\n", " print(f\"āš ļø **Error:** {health_check['error']}\")\n", " print(\"\\nšŸ”§ **Troubleshooting:**\")\n", " print(\"1. Check if Foundry Local is running: foundry service status\")\n", " print(\"2. Start a model: foundry model run phi-4-mini\")\n", " print(\"3. Verify the endpoint URL and port\")\n", " \n", " return health_check\n", "\n", "# Run the connection test\n", "connection_result = await test_service_connection()" ] }, { "cell_type": "markdown", "id": "3be87081", "metadata": {}, "source": [ "## Chainlit Application Structure\n", "\n", "Now let's examine the key components of our Chainlit application:" ] }, { "cell_type": "code", "execution_count": 18, "id": "5fe8c2e0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸ“„ **Chainlit Application Code:**\n", "============================================================\n", "#!/usr/bin/env python3\n", "# Copyright (c) Microsoft Corporation. All rights reserved.\n", "# Licensed under the MIT License.\n", "\n", "import os\n", "import chainlit as cl\n", "from openai import OpenAI\n", "\n", "try:\n", " from foundry_local import FoundryLocalManager\n", " FOUNDRY_SDK_AVAILABLE = True\n", "except ImportError:\n", " FOUNDRY_SDK_AVAILABLE = False\n", "\n", "# Global variables for client and model\n", "client = None\n", "model_name = None\n", "\n", "\n", "async def initialize_client():\n", " \"\"\"Initialize OpenAI client with Foundry Local or fallback configuration.\"\"\"\n", " global client, model_name\n", " \n", " alias = os.environ.get(\"MODEL\", \"phi-4-mini\")\n", " \n", " if FOUNDRY_SDK_AVAILABLE:\n", " try:\n", " # Use FoundryLocalManager for proper service management\n", " manager = FoundryLocalManager(alias)\n", " model_info = manager.get_model_info(alias)\n", " \n", " # Configure OpenAI client to use local Foundry service\n", " client = OpenAI(\n", " base_url=manager.endpoint,\n", " api_key=manager.api_key or \"not-required\" # Ensure API key is not None\n", " )\n", " model_name = model_info.id if model_info else alias\n", " print(f\"Initialized Foundry Local with model: {model_name}\")\n", " return True\n", " except Exception as e:\n", " print(f\"Warning: Could not use Foundry SDK ({e}), falling back to manual configuration\")\n", " \n", " # Fallback to manual configuration\n", " base_url = os.environ.get(\"BASE_URL\", \"http://localhost:51211\")\n", " api_key = os.environ.get(\"API_KEY\", \"not-required\")\n", " model_name = alias\n", " \n", " client = OpenAI(\n", " base_url=f\"{base_url}/v1\",\n", " api_key=api_key\n", " )\n", " print(f\"Initialized manual configuration with model: {model_name}\")\n", " return True\n", "\n", "\n", "@cl.on_chat_start\n", "async def start():\n", " \"\"\"Initialize the chat session.\"\"\"\n", " global client, model_name\n", " \n", " if client is None:\n", " try:\n", " await initialize_client()\n", " except Exception as e:\n", " error_msg = f\"āŒ **Initialization Error**\\n\\nCould not initialize the AI client. Please ensure Foundry Local is running.\\n\\n**Error:** {str(e)}\"\n", " await cl.Message(content=error_msg).send()\n", " return\n", " \n", " welcome_msg = f\"\"\"šŸ¤– **Welcome to Foundry Local RAG Chat!**\n", " \n", "**Model:** {model_name or 'Unknown'}\n", "**Powered by:** Microsoft Foundry Local\n", "\n", "You can ask me anything and I'll respond using the local AI model. The conversation supports:\n", "- āœ… Natural language processing\n", "- āœ… Code generation and explanation\n", "- āœ… Question answering\n", "- āœ… Creative writing\n", "\n", "Try asking me something!\"\"\"\n", " \n", " await cl.Message(content=welcome_msg).send()\n", "\n", "\n", "@cl.on_message\n", "async def main(message: cl.Message):\n", " \"\"\"Handle incoming messages and generate responses.\"\"\"\n", " global client, model_name\n", " \n", " if client is None:\n", " await cl.Message(content=\"āŒ Error: Client not initialized. Please restart the application.\").send()\n", " return\n", " \n", " try:\n", " # Show typing indicator\n", " msg = cl.Message(content=\"\")\n", " await msg.send()\n", " \n", " # Create streaming response\n", " stream = client.chat.completions.create(\n", " model=model_name,\n", " messages=[\n", " {\"role\": \"system\", \"content\": \"You are a helpful AI assistant powered by Microsoft Foundry Local. Provide clear, accurate, and helpful responses.\"},\n", " {\"role\": \"user\", \"content\": message.content}\n", " ],\n", " max_tokens=500,\n", " temperature=0.7,\n", " stream=True\n", " )\n", " \n", " # Stream the response\n", " for chunk in stream:\n", " if hasattr(chunk, 'choices') and len(chunk.choices) > 0:\n", " delta_content = chunk.choices[0].delta.content\n", " if delta_content is not None:\n", " await msg.stream_token(delta_content)\n", " \n", " # Finalize the message\n", " await msg.update()\n", " \n", " except Exception as e:\n", " error_msg = f\"āŒ **Error generating response:**\\n\\n{str(e)}\\n\\nšŸ’” **Troubleshooting:**\\n1. Ensure Foundry Local is running: `foundry service status`\\n2. Check if model is loaded: `foundry service ps`\\n3. Verify endpoint: `curl http://localhost:51211/v1/models`\"\n", " await cl.Message(content=error_msg).send()\n", "\n", "\n", "# Note: Client initialization happens in @cl.on_chat_start to ensure async context\n", "\n" ] } ], "source": [ "# Read the actual Chainlit application file\n", "app_file_path = \"../04/app.py\"\n", "\n", "try:\n", " with open(app_file_path, 'r', encoding='utf-8') as f:\n", " app_content = f.read()\n", " \n", " print(\"šŸ“„ **Chainlit Application Code:**\")\n", " print(\"=\" * 60)\n", " print(app_content)\n", " \n", "except FileNotFoundError:\n", " print(f\"āŒ Application file not found at {app_file_path}\")\n", " print(\"Let's create a sample application structure instead:\")\n", " \n", " sample_app = '''\n", "# Chainlit Application Structure\n", "\n", "import chainlit as cl\n", "from openai import AsyncOpenAI\n", "from foundry_local import FoundryLocalManager\n", "\n", "# Global client variable\n", "client = None\n", "\n", "@cl.on_chat_start\n", "async def on_chat_start():\n", " \"\"\"Initialize the chat session.\"\"\"\n", " # Setup client and welcome user\n", " \n", "@cl.on_message\n", "async def on_message(message: cl.Message):\n", " \"\"\"Handle incoming messages with streaming response.\"\"\"\n", " # Process message and stream response\n", " \n", "# Error handling and client setup functions...\n", "'''\n", " print(sample_app)" ] }, { "cell_type": "markdown", "id": "6e5674f9", "metadata": {}, "source": [ "## Key Chainlit Concepts\n", "\n", "Let's understand the main concepts used in Chainlit applications:" ] }, { "cell_type": "code", "execution_count": 19, "id": "ab85a613", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸŽÆ **Key Chainlit Concepts**\n", "==================================================\n", "\n", "šŸ”§ **@cl.on_chat_start**\n", " Purpose: Decorator for session initialization\n", " When Called: When a new chat session begins\n", " Typical Use: Setup client, show welcome message, initialize context\n", "\n", "šŸ”§ **@cl.on_message**\n", " Purpose: Decorator for message handling\n", " When Called: When user sends a message\n", " Typical Use: Process user input, generate AI response, stream output\n", "\n", "šŸ”§ **cl.Message**\n", " Purpose: Message object containing user input\n", " Properties: content, author, timestamp, elements\n", " Typical Use: Access user's message content and metadata\n", "\n", "šŸ”§ **cl.make_async**\n", " Purpose: Convert sync functions to async\n", " When Needed: When using sync OpenAI client in async context\n", " Typical Use: Wrap synchronous API calls for Chainlit compatibility\n", "\n", "šŸ”§ **Streaming Response**\n", " Purpose: Real-time message updates\n", " Implementation: Create empty message, update content progressively\n", " Typical Use: Better UX for long responses, real-time feedback\n" ] } ], "source": [ "print(\"šŸŽÆ **Key Chainlit Concepts**\")\n", "print(\"=\" * 50)\n", "\n", "chainlit_concepts = {\n", " \"@cl.on_chat_start\": {\n", " \"purpose\": \"Decorator for session initialization\",\n", " \"when_called\": \"When a new chat session begins\",\n", " \"typical_use\": \"Setup client, show welcome message, initialize context\"\n", " },\n", " \"@cl.on_message\": {\n", " \"purpose\": \"Decorator for message handling\",\n", " \"when_called\": \"When user sends a message\",\n", " \"typical_use\": \"Process user input, generate AI response, stream output\"\n", " },\n", " \"cl.Message\": {\n", " \"purpose\": \"Message object containing user input\",\n", " \"properties\": \"content, author, timestamp, elements\",\n", " \"typical_use\": \"Access user's message content and metadata\"\n", " },\n", " \"cl.make_async\": {\n", " \"purpose\": \"Convert sync functions to async\",\n", " \"when_needed\": \"When using sync OpenAI client in async context\",\n", " \"typical_use\": \"Wrap synchronous API calls for Chainlit compatibility\"\n", " },\n", " \"Streaming Response\": {\n", " \"purpose\": \"Real-time message updates\",\n", " \"implementation\": \"Create empty message, update content progressively\",\n", " \"typical_use\": \"Better UX for long responses, real-time feedback\"\n", " }\n", "}\n", "\n", "for concept, details in chainlit_concepts.items():\n", " print(f\"\\nšŸ”§ **{concept}**\")\n", " for key, value in details.items():\n", " print(f\" {key.replace('_', ' ').title()}: {value}\")" ] }, { "cell_type": "markdown", "id": "a36bd257", "metadata": {}, "source": [ "## Streaming Response Implementation\n", "\n", "Here's how streaming responses work in Chainlit:" ] }, { "cell_type": "code", "execution_count": 20, "id": "f70bd7a0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸŒŠ **Streaming Response Pattern**\n", "==================================================\n", "šŸ“ **Streaming Implementation:**\n", "\n", "# 1. Create an empty message to update progressively\n", "msg = cl.Message(content=\"\")\n", "await msg.send()\n", "\n", "# 2. Make streaming API call\n", "stream = await client.chat.completions.create(\n", " model=model_name,\n", " messages=messages,\n", " stream=True,\n", " max_tokens=1000\n", ")\n", "\n", "# 3. Process each chunk and update the message\n", "async for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " await msg.stream_token(chunk.choices[0].delta.content)\n", "\n", "# 4. Finalize the message\n", "await msg.update()\n", "\n", "\n", "āœØ **Benefits of Streaming:**\n", " šŸš€ **Real-time feedback**: Users see responses as they're generated\n", " āš” **Better perceived performance**: Feels faster than waiting for complete response\n", " šŸŽÆ **User engagement**: Keeps users engaged during long responses\n", " šŸ›‘ **Early termination**: Users can interrupt if response goes off-track\n", " šŸ’” **Professional UX**: Modern chat interface experience\n", "\n", "šŸ”§ **Implementation Notes:**\n", " ā€¢ Always use AsyncOpenAI for Chainlit applications\n", " ā€¢ Handle streaming errors gracefully with try-catch blocks\n", " ā€¢ Check for None content in delta chunks\n", " ā€¢ Update message when streaming is complete\n", " ā€¢ Consider rate limiting for production deployments\n", "\n", "==================================================\n", "šŸ“ **Streaming Implementation:**\n", "\n", "# 1. Create an empty message to update progressively\n", "msg = cl.Message(content=\"\")\n", "await msg.send()\n", "\n", "# 2. Make streaming API call\n", "stream = await client.chat.completions.create(\n", " model=model_name,\n", " messages=messages,\n", " stream=True,\n", " max_tokens=1000\n", ")\n", "\n", "# 3. Process each chunk and update the message\n", "async for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " await msg.stream_token(chunk.choices[0].delta.content)\n", "\n", "# 4. Finalize the message\n", "await msg.update()\n", "\n", "\n", "āœØ **Benefits of Streaming:**\n", " šŸš€ **Real-time feedback**: Users see responses as they're generated\n", " āš” **Better perceived performance**: Feels faster than waiting for complete response\n", " šŸŽÆ **User engagement**: Keeps users engaged during long responses\n", " šŸ›‘ **Early termination**: Users can interrupt if response goes off-track\n", " šŸ’” **Professional UX**: Modern chat interface experience\n", "\n", "šŸ”§ **Implementation Notes:**\n", " ā€¢ Always use AsyncOpenAI for Chainlit applications\n", " ā€¢ Handle streaming errors gracefully with try-catch blocks\n", " ā€¢ Check for None content in delta chunks\n", " ā€¢ Update message when streaming is complete\n", " ā€¢ Consider rate limiting for production deployments\n" ] } ], "source": [ "async def demonstrate_streaming_pattern():\n", " \"\"\"Demonstrate the streaming response pattern used in Chainlit.\"\"\"\n", " print(\"šŸŒŠ **Streaming Response Pattern**\")\n", " print(\"=\" * 50)\n", " \n", " # This is how streaming works in the actual Chainlit app\n", " streaming_code = '''\n", "# 1. Create an empty message to update progressively\n", "msg = cl.Message(content=\"\")\n", "await msg.send()\n", "\n", "# 2. Make streaming API call\n", "stream = await client.chat.completions.create(\n", " model=model_name,\n", " messages=messages,\n", " stream=True,\n", " max_tokens=1000\n", ")\n", "\n", "# 3. Process each chunk and update the message\n", "async for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " await msg.stream_token(chunk.choices[0].delta.content)\n", "\n", "# 4. Finalize the message\n", "await msg.update()\n", "'''\n", " \n", " print(\"šŸ“ **Streaming Implementation:**\")\n", " print(streaming_code)\n", " \n", " print(\"\\nāœØ **Benefits of Streaming:**\")\n", " benefits = [\n", " \"šŸš€ **Real-time feedback**: Users see responses as they're generated\",\n", " \"āš” **Better perceived performance**: Feels faster than waiting for complete response\",\n", " \"šŸŽÆ **User engagement**: Keeps users engaged during long responses\",\n", " \"šŸ›‘ **Early termination**: Users can interrupt if response goes off-track\",\n", " \"šŸ’” **Professional UX**: Modern chat interface experience\"\n", " ]\n", " \n", " for benefit in benefits:\n", " print(f\" {benefit}\")\n", " \n", " print(\"\\nšŸ”§ **Implementation Notes:**\")\n", " notes = [\n", " \"Always use AsyncOpenAI for Chainlit applications\",\n", " \"Handle streaming errors gracefully with try-catch blocks\",\n", " \"Check for None content in delta chunks\",\n", " \"Update message when streaming is complete\",\n", " \"Consider rate limiting for production deployments\"\n", " ]\n", " \n", " for note in notes:\n", " print(f\" ā€¢ {note}\")\n", "\n", "await demonstrate_streaming_pattern()" ] }, { "cell_type": "markdown", "id": "722fd61b", "metadata": {}, "source": [ "## Error Handling Patterns\n", "\n", "Robust error handling is crucial for production Chainlit applications:" ] }, { "cell_type": "code", "execution_count": 21, "id": "a723b71e", "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸ›”ļø **Error Handling Patterns**\n", "==================================================\n", "\n", "āš ļø **Client Initialization Failure**\n", " šŸ” Cause: Foundry Local service not running\n", " šŸ”§ Handling: Graceful fallback to manual configuration\n", " šŸ’¬ User Message: 'Service initializing, please wait...'\n", "\n", "āš ļø **Model Not Available**\n", " šŸ” Cause: Requested model not loaded\n", " šŸ”§ Handling: Try alternative models or suggest model loading\n", " šŸ’¬ User Message: 'Model unavailable, trying alternative...'\n", "\n", "āš ļø **Network Connection Error**\n", " šŸ” Cause: Network issues or service down\n", " šŸ”§ Handling: Retry with exponential backoff\n", " šŸ’¬ User Message: 'Connection issue, retrying...'\n", "\n", "āš ļø **Streaming Interruption**\n", " šŸ” Cause: Stream ends unexpectedly\n", " šŸ”§ Handling: Complete partial response gracefully\n", " šŸ’¬ User Message: 'Response completed (partial)'\n", "\n", "āš ļø **Rate Limiting**\n", " šŸ” Cause: Too many requests\n", " šŸ”§ Handling: Queue requests or ask user to wait\n", " šŸ’¬ User Message: 'High traffic, please wait a moment...'\n", "\n", "šŸ“‹ **Error Handling Best Practices:**\n", " šŸŽÆ **User-Friendly Messages**: Never show technical errors to users\n", " šŸ”„ **Automatic Retry**: Implement retry logic for transient failures\n", " šŸ“Š **Logging**: Log errors for debugging while keeping user experience smooth\n", " šŸ› ļø **Graceful Degradation**: Provide limited functionality when services are down\n", " šŸ’” **Helpful Suggestions**: Guide users on how to resolve issues\n", " āš” **Fast Failure**: Fail quickly rather than letting users wait indefinitely\n", "\n", "šŸ’» **Example Error Handling Code:**\n", "\n", "try:\n", " stream = await client.chat.completions.create(\n", " model=model_name,\n", " messages=messages,\n", " stream=True,\n", " max_tokens=1000\n", " )\n", " \n", " async for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " await msg.stream_token(chunk.choices[0].delta.content)\n", " \n", "except Exception as e:\n", " error_msg = \"I encountered an issue. Please try again.\"\n", " await cl.Message(content=error_msg, author=\"System\").send()\n", " # Log the actual error for debugging\n", " print(f\"Error: {e}\")\n", "\n" ] } ], "source": [ "async def demonstrate_error_handling():\n", " \"\"\"Show error handling patterns for Chainlit applications.\"\"\"\n", " print(\"šŸ›”ļø **Error Handling Patterns**\")\n", " print(\"=\" * 50)\n", " \n", " error_scenarios = {\n", " \"Client Initialization Failure\": {\n", " \"cause\": \"Foundry Local service not running\",\n", " \"handling\": \"Graceful fallback to manual configuration\",\n", " \"user_message\": \"Service initializing, please wait...\"\n", " },\n", " \"Model Not Available\": {\n", " \"cause\": \"Requested model not loaded\",\n", " \"handling\": \"Try alternative models or suggest model loading\",\n", " \"user_message\": \"Model unavailable, trying alternative...\"\n", " },\n", " \"Network Connection Error\": {\n", " \"cause\": \"Network issues or service down\",\n", " \"handling\": \"Retry with exponential backoff\",\n", " \"user_message\": \"Connection issue, retrying...\"\n", " },\n", " \"Streaming Interruption\": {\n", " \"cause\": \"Stream ends unexpectedly\",\n", " \"handling\": \"Complete partial response gracefully\",\n", " \"user_message\": \"Response completed (partial)\"\n", " },\n", " \"Rate Limiting\": {\n", " \"cause\": \"Too many requests\",\n", " \"handling\": \"Queue requests or ask user to wait\",\n", " \"user_message\": \"High traffic, please wait a moment...\"\n", " }\n", " }\n", " \n", " for scenario, details in error_scenarios.items():\n", " print(f\"\\nāš ļø **{scenario}**\")\n", " print(f\" šŸ” Cause: {details['cause']}\")\n", " print(f\" šŸ”§ Handling: {details['handling']}\")\n", " print(f\" šŸ’¬ User Message: '{details['user_message']}'\")\n", " \n", " print(\"\\nšŸ“‹ **Error Handling Best Practices:**\")\n", " best_practices = [\n", " \"šŸŽÆ **User-Friendly Messages**: Never show technical errors to users\",\n", " \"šŸ”„ **Automatic Retry**: Implement retry logic for transient failures\",\n", " \"šŸ“Š **Logging**: Log errors for debugging while keeping user experience smooth\",\n", " \"šŸ› ļø **Graceful Degradation**: Provide limited functionality when services are down\",\n", " \"šŸ’” **Helpful Suggestions**: Guide users on how to resolve issues\",\n", " \"āš” **Fast Failure**: Fail quickly rather than letting users wait indefinitely\"\n", " ]\n", " \n", " for practice in best_practices:\n", " print(f\" {practice}\")\n", " \n", " # Show example error handling code\n", " print(\"\\nšŸ’» **Example Error Handling Code:**\")\n", " error_code = '''\n", "try:\n", " stream = await client.chat.completions.create(\n", " model=model_name,\n", " messages=messages,\n", " stream=True,\n", " max_tokens=1000\n", " )\n", " \n", " async for chunk in stream:\n", " if chunk.choices[0].delta.content is not None:\n", " await msg.stream_token(chunk.choices[0].delta.content)\n", " \n", "except Exception as e:\n", " error_msg = \"I encountered an issue. Please try again.\"\n", " await cl.Message(content=error_msg, author=\"System\").send()\n", " # Log the actual error for debugging\n", " print(f\"Error: {e}\")\n", "'''\n", " print(error_code)\n", "\n", "await demonstrate_error_handling()" ] }, { "cell_type": "markdown", "id": "6269dfef", "metadata": {}, "source": [ "## Running the Chainlit Application\n", "\n", "Here's how to run and deploy the Chainlit application:" ] }, { "cell_type": "code", "execution_count": 22, "id": "60fa850b", "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸš€ **Running Chainlit Application**\n", "==================================================\n", "\n", "šŸ”§ **Development Mode**\n", " Command: chainlit run app.py --watch\n", " Description: Auto-reload on file changes\n", " Port: 8000 (default)\n", " Use_Case: Local development and testing\n", "\n", "šŸ”§ **Production Mode**\n", " Command: chainlit run app.py --host 0.0.0.0 --port 8080\n", " Description: Production deployment\n", " Port: 8080 (configurable)\n", " Use_Case: Server deployment\n", "\n", "šŸ”§ **Custom Configuration**\n", " Command: chainlit run app.py -h 0.0.0.0 -p 3000 --no-cache\n", " Description: Custom host, port, and caching options\n", " Port: 3000 (custom)\n", " Use_Case: Specific deployment requirements\n", "\n", "šŸŒ **Access Points:**\n", " šŸ“± **Local Development**: http://localhost:8000\n", " šŸŒ **Network Access**: http://YOUR_IP:8000\n", " šŸ”— **Production**: https://your-domain.com\n", " šŸ“Š **Health Check**: Add /health endpoint for monitoring\n", "\n", "āš™ļø **Environment Variables for Production:**\n", " BASE_URL=http://localhost:8000 (Foundry Local endpoint)\n", " API_KEY=your-api-key (if required)\n", " MODEL_NAME=phi-4-mini (default model)\n", " MAX_TOKENS=1000 (response length limit)\n", " CHAINLIT_HOST=0.0.0.0 (production host)\n", " CHAINLIT_PORT=8080 (production port)\n", "\n", "==================================================\n", "\n", "šŸ”§ **Development Mode**\n", " Command: chainlit run app.py --watch\n", " Description: Auto-reload on file changes\n", " Port: 8000 (default)\n", " Use_Case: Local development and testing\n", "\n", "šŸ”§ **Production Mode**\n", " Command: chainlit run app.py --host 0.0.0.0 --port 8080\n", " Description: Production deployment\n", " Port: 8080 (configurable)\n", " Use_Case: Server deployment\n", "\n", "šŸ”§ **Custom Configuration**\n", " Command: chainlit run app.py -h 0.0.0.0 -p 3000 --no-cache\n", " Description: Custom host, port, and caching options\n", " Port: 3000 (custom)\n", " Use_Case: Specific deployment requirements\n", "\n", "šŸŒ **Access Points:**\n", " šŸ“± **Local Development**: http://localhost:8000\n", " šŸŒ **Network Access**: http://YOUR_IP:8000\n", " šŸ”— **Production**: https://your-domain.com\n", " šŸ“Š **Health Check**: Add /health endpoint for monitoring\n", "\n", "āš™ļø **Environment Variables for Production:**\n", " BASE_URL=http://localhost:8000 (Foundry Local endpoint)\n", " API_KEY=your-api-key (if required)\n", " MODEL_NAME=phi-4-mini (default model)\n", " MAX_TOKENS=1000 (response length limit)\n", " CHAINLIT_HOST=0.0.0.0 (production host)\n", " CHAINLIT_PORT=8080 (production port)\n" ] } ], "source": [ "print(\"šŸš€ **Running Chainlit Application**\")\n", "print(\"=\" * 50)\n", "\n", "running_instructions = {\n", " \"Development Mode\": {\n", " \"command\": \"chainlit run app.py --watch\",\n", " \"description\": \"Auto-reload on file changes\",\n", " \"port\": \"8000 (default)\",\n", " \"use_case\": \"Local development and testing\"\n", " },\n", " \"Production Mode\": {\n", " \"command\": \"chainlit run app.py --host 0.0.0.0 --port 8080\",\n", " \"description\": \"Production deployment\",\n", " \"port\": \"8080 (configurable)\",\n", " \"use_case\": \"Server deployment\"\n", " },\n", " \"Custom Configuration\": {\n", " \"command\": \"chainlit run app.py -h 0.0.0.0 -p 3000 --no-cache\",\n", " \"description\": \"Custom host, port, and caching options\",\n", " \"port\": \"3000 (custom)\",\n", " \"use_case\": \"Specific deployment requirements\"\n", " }\n", "}\n", "\n", "for mode, config in running_instructions.items():\n", " print(f\"\\nšŸ”§ **{mode}**\")\n", " for key, value in config.items():\n", " print(f\" {key.title()}: {value}\")\n", "\n", "print(\"\\nšŸŒ **Access Points:**\")\n", "access_info = [\n", " \"šŸ“± **Local Development**: http://localhost:8000\",\n", " \"šŸŒ **Network Access**: http://YOUR_IP:8000\",\n", " \"šŸ”— **Production**: https://your-domain.com\",\n", " \"šŸ“Š **Health Check**: Add /health endpoint for monitoring\"\n", "]\n", "\n", "for info in access_info:\n", " print(f\" {info}\")\n", "\n", "print(\"\\nāš™ļø **Environment Variables for Production:**\")\n", "env_vars = {\n", " \"BASE_URL\": \"http://localhost:8000 (Foundry Local endpoint)\",\n", " \"API_KEY\": \"your-api-key (if required)\",\n", " \"MODEL_NAME\": \"phi-4-mini (default model)\",\n", " \"MAX_TOKENS\": \"1000 (response length limit)\",\n", " \"CHAINLIT_HOST\": \"0.0.0.0 (production host)\",\n", " \"CHAINLIT_PORT\": \"8080 (production port)\"\n", "}\n", "\n", "for var, desc in env_vars.items():\n", " print(f\" {var}={desc}\")" ] }, { "cell_type": "markdown", "id": "928e0dc1", "metadata": {}, "source": [ "## Customization Options\n", "\n", "Chainlit provides extensive customization options:" ] }, { "cell_type": "code", "execution_count": 23, "id": "8e0ecc0c", "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸŽØ **Chainlit Customization Options**\n", "==================================================\n", "\n", "šŸ”§ **UI Customization**\n", " šŸ“„ Config: .chainlit/config.toml\n", " āš™ļø Options:\n", " ā€¢ Custom theme colors and fonts\n", " ā€¢ Company logo and branding\n", " ā€¢ Custom CSS styling\n", " ā€¢ Dark/light mode preferences\n", "\n", "šŸ”§ **Chat Features**\n", " šŸ“„ Config: app.py (programmatic)\n", " āš™ļø Options:\n", " ā€¢ File upload support\n", " ā€¢ Image and media handling\n", " ā€¢ Custom message elements\n", " ā€¢ Action buttons and quick replies\n", "\n", "šŸ”§ **Authentication**\n", " šŸ“„ Config: auth.py + config.toml\n", " āš™ļø Options:\n", " ā€¢ OAuth integration (Google, GitHub)\n", " ā€¢ LDAP/Active Directory\n", " ā€¢ Custom authentication providers\n", " ā€¢ Role-based access control\n", "\n", "šŸ”§ **Deployment**\n", " šŸ“„ Config: docker-compose.yml / Dockerfile\n", " āš™ļø Options:\n", " ā€¢ Docker containerization\n", " ā€¢ Kubernetes deployment\n", " ā€¢ Cloud platform integration\n", " ā€¢ Reverse proxy configuration\n", "\n", "šŸ“ **Sample .chainlit/config.toml:**\n", "\n", "[project]\n", "name = \"Foundry Local Chat\"\n", "author = \"Your Organization\"\n", "description = \"AI Chat powered by Foundry Local\"\n", "\n", "[UI]\n", "name = \"Foundry AI Assistant\"\n", "show_readme_as_default = true\n", "show_cloud_icon = false\n", "\n", "[theme]\n", "primary_color = \"#0078d4\"\n", "background_color = \"#ffffff\"\n", "text_color = \"#323130\"\n", "\n", "[features]\n", "allow_unsafe_html = false\n", "max_message_size = 4096\n", "max_file_size_mb = 10\n", "\n" ] } ], "source": [ "print(\"šŸŽØ **Chainlit Customization Options**\")\n", "print(\"=\" * 50)\n", "\n", "customization_areas = {\n", " \"UI Customization\": {\n", " \"config_file\": \".chainlit/config.toml\",\n", " \"options\": [\n", " \"Custom theme colors and fonts\",\n", " \"Company logo and branding\",\n", " \"Custom CSS styling\",\n", " \"Dark/light mode preferences\"\n", " ]\n", " },\n", " \"Chat Features\": {\n", " \"config_file\": \"app.py (programmatic)\",\n", " \"options\": [\n", " \"File upload support\",\n", " \"Image and media handling\",\n", " \"Custom message elements\",\n", " \"Action buttons and quick replies\"\n", " ]\n", " },\n", " \"Authentication\": {\n", " \"config_file\": \"auth.py + config.toml\",\n", " \"options\": [\n", " \"OAuth integration (Google, GitHub)\",\n", " \"LDAP/Active Directory\",\n", " \"Custom authentication providers\",\n", " \"Role-based access control\"\n", " ]\n", " },\n", " \"Deployment\": {\n", " \"config_file\": \"docker-compose.yml / Dockerfile\",\n", " \"options\": [\n", " \"Docker containerization\",\n", " \"Kubernetes deployment\",\n", " \"Cloud platform integration\",\n", " \"Reverse proxy configuration\"\n", " ]\n", " }\n", "}\n", "\n", "for area, details in customization_areas.items():\n", " print(f\"\\nšŸ”§ **{area}**\")\n", " print(f\" šŸ“„ Config: {details['config_file']}\")\n", " print(f\" āš™ļø Options:\")\n", " for option in details['options']:\n", " print(f\" ā€¢ {option}\")\n", "\n", "# Show sample configuration\n", "print(\"\\nšŸ“ **Sample .chainlit/config.toml:**\")\n", "sample_config = '''\n", "[project]\n", "name = \"Foundry Local Chat\"\n", "author = \"Your Organization\"\n", "description = \"AI Chat powered by Foundry Local\"\n", "\n", "[UI]\n", "name = \"Foundry AI Assistant\"\n", "show_readme_as_default = true\n", "show_cloud_icon = false\n", "\n", "[theme]\n", "primary_color = \"#0078d4\"\n", "background_color = \"#ffffff\"\n", "text_color = \"#323130\"\n", "\n", "[features]\n", "allow_unsafe_html = false\n", "max_message_size = 4096\n", "max_file_size_mb = 10\n", "'''\n", "print(sample_config)" ] }, { "cell_type": "markdown", "id": "b33562e8", "metadata": {}, "source": [ "## Advanced Features\n", "\n", "Explore advanced Chainlit features for production applications:" ] }, { "cell_type": "code", "execution_count": 24, "id": "86670e5d", "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "šŸš€ **Advanced Chainlit Features**\n", "==================================================\n", "\n", "šŸŽÆ **Session Management**\n", " šŸ“ Description: Maintain conversation context across messages\n", " šŸ”§ Implementation: cl.user_session for storing state\n", " šŸ’” Use Cases:\n", " ā€¢ Multi-turn conversations\n", " ā€¢ User preferences\n", " ā€¢ Context persistence\n", "\n", "šŸŽÆ **File Uploads**\n", " šŸ“ Description: Handle document uploads and processing\n", " šŸ”§ Implementation: @cl.on_file_upload decorator\n", " šŸ’” Use Cases:\n", " ā€¢ Document analysis\n", " ā€¢ Image processing\n", " ā€¢ Data ingestion\n", "\n", "šŸŽÆ **Action Buttons**\n", " šŸ“ Description: Interactive buttons for user actions\n", " šŸ”§ Implementation: cl.Action elements\n", " šŸ’” Use Cases:\n", " ā€¢ Quick replies\n", " ā€¢ Workflow triggers\n", " ā€¢ Menu navigation\n", "\n", "šŸŽÆ **Data Persistence**\n", " šŸ“ Description: Store conversation history and user data\n", " šŸ”§ Implementation: Database integration\n", " šŸ’” Use Cases:\n", " ā€¢ Chat history\n", " ā€¢ User analytics\n", " ā€¢ Feedback collection\n", "\n", "šŸŽÆ **Multi-modal Support**\n", " šŸ“ Description: Handle text, images, and other media\n", " šŸ”§ Implementation: cl.Image, cl.File elements\n", " šŸ’” Use Cases:\n", " ā€¢ Visual Q&A\n", " ā€¢ Document chat\n", " ā€¢ Media analysis\n", "\n", "šŸ’» **Session Management Example:**\n", "\n", "@cl.on_chat_start\n", "async def on_chat_start():\n", " # Initialize session state\n", " cl.user_session.set(\"conversation_history\", [])\n", " cl.user_session.set(\"user_preferences\", {\"temperature\": 0.7})\n", "\n", "@cl.on_message\n", "async def on_message(message: cl.Message):\n", " # Get session state\n", " history = cl.user_session.get(\"conversation_history\", [])\n", " preferences = cl.user_session.get(\"user_preferences\", {})\n", " \n", " # Add current message to history\n", " history.append({\"role\": \"user\", \"content\": message.content})\n", " \n", " # Use full conversation context\n", " response = await client.chat.completions.create(\n", " model=\"phi-4-mini\",\n", " messages=history,\n", " temperature=preferences.get(\"temperature\", 0.7)\n", " )\n", " \n", " # Update session with AI response\n", " history.append({\"role\": \"assistant\", \"content\": response.choices[0].message.content})\n", " cl.user_session.set(\"conversation_history\", history)\n", "\n" ] } ], "source": [ "print(\"šŸš€ **Advanced Chainlit Features**\")\n", "print(\"=\" * 50)\n", "\n", "advanced_features = {\n", " \"Session Management\": {\n", " \"description\": \"Maintain conversation context across messages\",\n", " \"implementation\": \"cl.user_session for storing state\",\n", " \"use_cases\": [\"Multi-turn conversations\", \"User preferences\", \"Context persistence\"]\n", " },\n", " \"File Uploads\": {\n", " \"description\": \"Handle document uploads and processing\",\n", " \"implementation\": \"@cl.on_file_upload decorator\",\n", " \"use_cases\": [\"Document analysis\", \"Image processing\", \"Data ingestion\"]\n", " },\n", " \"Action Buttons\": {\n", " \"description\": \"Interactive buttons for user actions\",\n", " \"implementation\": \"cl.Action elements\",\n", " \"use_cases\": [\"Quick replies\", \"Workflow triggers\", \"Menu navigation\"]\n", " },\n", " \"Data Persistence\": {\n", " \"description\": \"Store conversation history and user data\",\n", " \"implementation\": \"Database integration\",\n", " \"use_cases\": [\"Chat history\", \"User analytics\", \"Feedback collection\"]\n", " },\n", " \"Multi-modal Support\": {\n", " \"description\": \"Handle text, images, and other media\",\n", " \"implementation\": \"cl.Image, cl.File elements\",\n", " \"use_cases\": [\"Visual Q&A\", \"Document chat\", \"Media analysis\"]\n", " }\n", "}\n", "\n", "for feature, details in advanced_features.items():\n", " print(f\"\\nšŸŽÆ **{feature}**\")\n", " print(f\" šŸ“ Description: {details['description']}\")\n", " print(f\" šŸ”§ Implementation: {details['implementation']}\")\n", " print(f\" šŸ’” Use Cases:\")\n", " for use_case in details['use_cases']:\n", " print(f\" ā€¢ {use_case}\")\n", "\n", "# Show example code for session management\n", "print(\"\\nšŸ’» **Session Management Example:**\")\n", "session_code = '''\n", "@cl.on_chat_start\n", "async def on_chat_start():\n", " # Initialize session state\n", " cl.user_session.set(\"conversation_history\", [])\n", " cl.user_session.set(\"user_preferences\", {\"temperature\": 0.7})\n", "\n", "@cl.on_message\n", "async def on_message(message: cl.Message):\n", " # Get session state\n", " history = cl.user_session.get(\"conversation_history\", [])\n", " preferences = cl.user_session.get(\"user_preferences\", {})\n", " \n", " # Add current message to history\n", " history.append({\"role\": \"user\", \"content\": message.content})\n", " \n", " # Use full conversation context\n", " response = await client.chat.completions.create(\n", " model=\"phi-4-mini\",\n", " messages=history,\n", " temperature=preferences.get(\"temperature\", 0.7)\n", " )\n", " \n", " # Update session with AI response\n", " history.append({\"role\": \"assistant\", \"content\": response.choices[0].message.content})\n", " cl.user_session.set(\"conversation_history\", history)\n", "'''\n", "print(session_code)" ] }, { "cell_type": "markdown", "id": "28be192a", "metadata": {}, "source": [ "## Production Deployment Checklist\n", "\n", "Essential considerations for deploying Chainlit applications in production:" ] }, { "cell_type": "code", "execution_count": 25, "id": "5b18750c", "metadata": { "vscode": { "languageId": "powershell" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "āœ… **Production Deployment Checklist**\n", "==================================================\n", "\n", "šŸ”’ Security\n", " ā˜ Enable authentication and authorization\n", " ā˜ Use HTTPS with proper SSL certificates\n", " ā˜ Implement rate limiting and request validation\n", " ā˜ Sanitize user inputs and prevent injection attacks\n", " ā˜ Set up proper CORS policies\n", " ā˜ Use environment variables for sensitive configuration\n", "\n", "āš” Performance\n", " ā˜ Configure connection pooling for database\n", " ā˜ Implement caching for frequent responses\n", " ā˜ Set up load balancing for multiple instances\n", " ā˜ Monitor memory usage and optimize where needed\n", " ā˜ Configure appropriate timeout values\n", " ā˜ Use CDN for static assets\n", "\n", "šŸ“Š Monitoring\n", " ā˜ Set up application logging and monitoring\n", " ā˜ Configure health checks and uptime monitoring\n", " ā˜ Track user engagement and conversation metrics\n", " ā˜ Monitor API response times and error rates\n", " ā˜ Set up alerting for critical issues\n", " ā˜ Implement user feedback collection\n", "\n", "šŸ› ļø Maintenance\n", " ā˜ Regular backups of conversation data\n", " ā˜ Automated deployment pipelines\n", " ā˜ Version control for configuration changes\n", " ā˜ Documentation for troubleshooting\n", " ā˜ Capacity planning and scaling procedures\n", " ā˜ Update procedures for dependencies\n", "\n", "šŸŒ Infrastructure\n", " ā˜ Container orchestration (Docker/Kubernetes)\n", " ā˜ Reverse proxy configuration (nginx/Apache)\n", " ā˜ Database setup and optimization\n", " ā˜ Network security and firewall rules\n", " ā˜ Backup and disaster recovery plans\n", " ā˜ Multi-region deployment for redundancy\n", "\n", "šŸš€ **Quick Production Setup Commands:**\n", "\n", "šŸ’” # Build Docker image\n", " docker build -t chainlit-app .\n", "\n", "\n", "šŸ’” # Run with production settings\n", " docker run -d -p 8080:8080 -e NODE_ENV=production chainlit-app\n", "\n", "\n", "šŸ’” # Health check\n", " curl http://localhost:8080/health\n", "\n", "\n", "šŸ’” # Monitor logs\n", " docker logs -f chainlit-app\n" ] } ], "source": [ "print(\"āœ… **Production Deployment Checklist**\")\n", "print(\"=\" * 50)\n", "\n", "deployment_checklist = {\n", " \"šŸ”’ Security\": [\n", " \"Enable authentication and authorization\",\n", " \"Use HTTPS with proper SSL certificates\",\n", " \"Implement rate limiting and request validation\",\n", " \"Sanitize user inputs and prevent injection attacks\",\n", " \"Set up proper CORS policies\",\n", " \"Use environment variables for sensitive configuration\"\n", " ],\n", " \"āš” Performance\": [\n", " \"Configure connection pooling for database\",\n", " \"Implement caching for frequent responses\",\n", " \"Set up load balancing for multiple instances\",\n", " \"Monitor memory usage and optimize where needed\",\n", " \"Configure appropriate timeout values\",\n", " \"Use CDN for static assets\"\n", " ],\n", " \"šŸ“Š Monitoring\": [\n", " \"Set up application logging and monitoring\",\n", " \"Configure health checks and uptime monitoring\",\n", " \"Track user engagement and conversation metrics\",\n", " \"Monitor API response times and error rates\",\n", " \"Set up alerting for critical issues\",\n", " \"Implement user feedback collection\"\n", " ],\n", " \"šŸ› ļø Maintenance\": [\n", " \"Regular backups of conversation data\",\n", " \"Automated deployment pipelines\",\n", " \"Version control for configuration changes\",\n", " \"Documentation for troubleshooting\",\n", " \"Capacity planning and scaling procedures\",\n", " \"Update procedures for dependencies\"\n", " ],\n", " \"šŸŒ Infrastructure\": [\n", " \"Container orchestration (Docker/Kubernetes)\",\n", " \"Reverse proxy configuration (nginx/Apache)\",\n", " \"Database setup and optimization\",\n", " \"Network security and firewall rules\",\n", " \"Backup and disaster recovery plans\",\n", " \"Multi-region deployment for redundancy\"\n", " ]\n", "}\n", "\n", "for category, items in deployment_checklist.items():\n", " print(f\"\\n{category}\")\n", " for item in items:\n", " print(f\" ā˜ {item}\")\n", "\n", "print(\"\\nšŸš€ **Quick Production Setup Commands:**\")\n", "commands = [\n", " \"# Build Docker image\",\n", " \"docker build -t chainlit-app .\",\n", " \"\",\n", " \"# Run with production settings\",\n", " \"docker run -d -p 8080:8080 -e NODE_ENV=production chainlit-app\",\n", " \"\",\n", " \"# Health check\",\n", " \"curl http://localhost:8080/health\",\n", " \"\",\n", " \"# Monitor logs\",\n", " \"docker logs -f chainlit-app\"\n", "]\n", "\n", "for cmd in commands:\n", " if cmd.startswith(\"#\"):\n", " print(f\"\\nšŸ’” {cmd}\")\n", " elif cmd:\n", " print(f\" {cmd}\")\n", " else:\n", " print()" ] }, { "cell_type": "markdown", "id": "beab2130", "metadata": {}, "source": [ "## Summary and Best Practices\n", "\n", "This notebook covered the complete Chainlit application development process:\n", "\n", "### āœ… Key Components Covered\n", "\n", "1. **šŸ”§ Client Setup**: Foundry Local SDK integration with fallback configuration\n", "2. **šŸŒŠ Streaming Responses**: Real-time message updates for better UX\n", "3. **šŸ›”ļø Error Handling**: Graceful failure handling and user-friendly messages\n", "4. **āš™ļø Configuration**: Environment-based setup and customization options\n", "5. **šŸš€ Deployment**: Production-ready deployment patterns and best practices\n", "\n", "### šŸŽÆ Chainlit Application Architecture\n", "\n", "```\n", "User Browser ā†ā†’ Chainlit UI ā†ā†’ Python Backend ā†ā†’ Foundry Local ā†ā†’ AI Model\n", " ā†“ ā†“ ā†“ ā†“ ā†“\n", " Web UI Event Handlers OpenAI Client HTTP API Local GPU\n", "```\n", "\n", "### šŸ’” Best Practices Summary\n", "\n", "- **šŸ”„ Always Use Async**: Chainlit requires async functions for non-blocking operations\n", "- **šŸŒŠ Implement Streaming**: Provides better user experience than waiting for complete responses\n", "- **šŸ›”ļø Handle Errors Gracefully**: Never show technical errors to end users\n", "- **šŸ“Š Monitor Performance**: Track response times and user engagement metrics\n", "- **šŸ”’ Secure by Default**: Implement authentication and input validation from the start\n", "- **āš” Optimize for Scale**: Design for multiple concurrent users from day one\n", "\n", "### šŸš€ Next Steps\n", "\n", "- **šŸ“± Multi-Modal Support**: Add image and document processing capabilities\n", "- **šŸ¤– Agent Integration**: Connect with multi-agent systems for complex workflows\n", "- **šŸ“Š Analytics Dashboard**: Build admin interfaces for monitoring and management\n", "- **šŸ”§ Custom Plugins**: Develop custom Chainlit elements and integrations\n", "- **šŸŒ API Integration**: Connect with external services and databases\n", "\n", "This Chainlit application demonstrates how to build production-ready conversational AI interfaces that leverage the power of local AI models through Microsoft Foundry Local while providing a modern, responsive user experience." ] } ], "metadata": { "kernelspec": { "display_name": "pydev", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.8" } }, "nbformat": 4, "nbformat_minor": 5 }