{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# 统一\n", "\n", "[统一](https://unify.ai/hub)动态地将每个查询路由到最佳的LLM，支持OpenAI、MistralAI、Perplexity AI和Together AI等提供商。您还可以使用单个API密钥访问所有提供商。\n", "\n", "您可以查看我们的[实时基准测试](https://unify.ai/hub/mixtral-8x7b-instruct-v0.1)以了解数据的来源！\n"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## 安装说明\n"]}, {"cell_type": "markdown", "metadata": {}, "source": ["首先，让我们安装LlamaIndex 🦙和Unify集成。\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["%pip install llama-index-llms-unify llama-index"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## 环境设置\n", "\n", "确保设置`UNIFY_API_KEY`环境变量。您可以在[Unify控制台](https://console.unify.ai/login)中获取密钥。\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["import os\n", "\n", "os.environ[\"UNIFY_API_KEY\"] = \"<YOUR API KEY>\""]}, {"cell_type": "markdown", "metadata": {}, "source": ["## 使用LlamaIndex与Unify\n", "\n", "LlamaIndex是一个用于索引和搜索文本数据的工具，而Unify是一个用于数据整合和清洗的工具。在这个示例中，我们将展示如何使用LlamaIndex和Unify来处理文本数据。\n"]}, {"cell_type": "markdown", "metadata": {}, "source": ["### 路由请求\n", "\n", "我们可以做的第一件事是初始化并查询一个聊天模型。要配置Unify的路由器，可以将一个端点字符串传递给 `Unify`。您可以在[Unify文档](https://unify.ai/docs/hub/concepts/runtime_routing.html)中了解更多信息。\n", "\n", "在这种情况下，我们将使用在输入成本方面最便宜的 `llama2-70b` 端点，然后使用 `complete`。\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [{"data": {"text/plain": ["CompletionResponse(text=\"  I'm doing well, thanks for asking! It's always a pleasure to chat with you. I hope you're having a great day too! Is there anything specific you'd like to talk about or ask me? I'm here to help with any questions you might have.\", additional_kwargs={}, raw={'id': 'meta-llama/Llama-2-70b-chat-hf-b90de288-1927-4f32-9ecb-368983c45321', 'choices': [Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=\"  I'm doing well, thanks for asking! It's always a pleasure to chat with you. I hope you're having a great day too! Is there anything specific you'd like to talk about or ask me? I'm here to help with any questions you might have.\", role='assistant', function_call=None, tool_calls=None, tool_call_id=None))], 'created': 1711047739, 'model': 'llama-2-70b-chat@anyscale', 'object': 'chat.completion', 'system_fingerprint': None, 'usage': CompletionUsage(completion_tokens=62, prompt_tokens=16, total_tokens=78, cost=7.8e-05)}, logprobs=None, delta=None)"]}, "execution_count": null, "metadata": {}, "output_type": "execute_result"}], "source": ["from llama_index.llms.unify import Unify\n", "\n", "llm = Unify(model=\"llama-2-70b-chat@dinput-cost\")\n", "llm.complete(\"How are you today, llama?\")"]}, {"cell_type": "markdown", "metadata": {}, "source": ["### 单点登录\n", "\n", "如果您不希望路由器选择提供者，您也可以使用我们的SSO在不与所有提供者建立帐户的情况下查询不同提供者的端点。例如，以下所有端点都是有效的：\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["llm = Unify(model=\"llama-2-70b-chat@together-ai\")\n", "llm = Unify(model=\"gpt-3.5-turbo@openai\")\n", "llm = Unify(model=\"mixtral-8x7b-instruct-v0.1@mistral-ai\")"]}, {"cell_type": "markdown", "metadata": {}, "source": ["这样可以让您快速切换和测试不同的模型和提供商。例如，如果您正在开发一个使用gpt-4作为核心的应用程序，您可以在开发和/或测试过程中使用这个功能来查询成本更低的LLM，以降低成本。\n", "\n", "在[这里](https://unify.ai/hub)查看可用的内容！\n"]}, {"cell_type": "markdown", "metadata": {}, "source": ["### 流式传输和优化延迟\n", "\n", "如果您正在构建一个对响应速度要求很高的应用程序，您很可能希望获得一个流式响应。此外，理想情况下，您应该使用具有最低“首个令牌时间”的提供者，以减少用户等待响应的时间。在Unify中，这看起来应该是这样的：\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["llm = Unify(model=\"mixtral-8x7b-instruct-v0.1@ttft\")\n", "\n", "response = llm.stream_complete(\n", "    \"Translate the following to German: \"\n", "    \"Hey, there's an emergency in translation street, \"\n", "    \"please send help asap!\"\n", ")"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Model and provider are : mixtral-8x7b-instruct-v0.1@mistral-ai\n", "\n", "Hallo, es gibt einen Notfall in der Übersetzungsstraße, bitte senden Sie Hilfe so schnell wie möglich!\n", "\n", "(Note: This is a literal translation and the term \"Übersetzungsstraße\" is not a standard or commonly used term in German. A more natural way to express the idea of a \"emergency in translation\" could be \"Notfall bei Übersetzungen\" or \"akute Übersetzungsnotwendigkeit\".)"]}], "source": ["show_provider = True\n", "for r in response:\n", "    if show_provider:\n", "        print(f\"Model and provider are : {r.raw['model']}\\n\")\n", "        show_provider = False\n", "    print(r.delta, end=\"\", flush=True)"]}, {"cell_type": "markdown", "metadata": {}, "source": ["### 异步调用和最低输入成本\n", "\n", "最后但并非最不重要的是，您还可以异步运行请求。对于长文档摘要等任务，优化输入成本至关重要。Unify的动态路由器也可以做到这一点！\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["Model and provider are : mixtral-8x7b-instruct-v0.1@deepinfra\n", "\n", " OpenAI: Pioneering 'safe' artificial general intelligence.\n"]}], "source": ["llm = Unify(model=\"mixtral-8x7b-instruct-v0.1@input-cost\")\n", "\n", "response = await llm.acomplete(\n", "    \"Summarize this in 10 words or less. OpenAI is a U.S. based artificial intelligence \"\n", "    \"(AI) research organization founded in December 2015, researching artificial intelligence \"\n", "    \"with the goal of developing 'safe and beneficial' artificial general intelligence, \"\n", "    \"which it defines as 'highly autonomous systems that outperform humans at most economically \"\n", "    \"valuable work'. As one of the leading organizations of the AI spring, it has developed \"\n", "    \"several large language models, advanced image generation models, and previously, released \"\n", "    \"open-source models. Its release of ChatGPT has been credited with starting the AI spring\"\n", ")\n", "\n", "print(f\"Model and provider are : {response.raw['model']}\\n\")\n", "print(response)"]}], "metadata": {"kernelspec": {"display_name": "base", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3"}}, "nbformat": 4, "nbformat_minor": 2}