{"cells": [{"cell_type": "markdown", "id": "c798b3ad", "metadata": {}, "source": ["<a href=\"https://colab.research.google.com/github/jerryjliu/llama_index/blob/main/docs/docs/examples/llm/vicuna.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"在 Colab 中打开\"/></a>\n"]}, {"cell_type": "markdown", "id": "2e33dced-e587-4397-81b3-d6606aa1738a", "metadata": {}, "source": ["# 复制 - Vicuna 13B\n"]}, {"cell_type": "markdown", "id": "5863dde9-84a0-4c33-ad52-cc767442f63f", "metadata": {}, "source": ["## 设置\n"]}, {"cell_type": "markdown", "id": "54194080", "metadata": {}, "source": ["如果您在colab上打开这个笔记本，您可能需要安装LlamaIndex 🦙。\n"]}, {"cell_type": "code", "execution_count": null, "id": "96fbd8f5", "metadata": {}, "outputs": [], "source": ["%pip install llama-index-llms-replicate"]}, {"cell_type": "code", "execution_count": null, "id": "7bbadca3", "metadata": {}, "outputs": [], "source": ["!pip install llama-index"]}, {"cell_type": "markdown", "id": "af971d61-2ed8-4265-85bf-019e86e550f8", "metadata": {}, "source": ["确保你已经设置了`REPLICATE_API_TOKEN`环境变量。  \n", "如果你还没有这个环境变量，可以前往 https://replicate.com/ 获取一个。\n"]}, {"cell_type": "code", "execution_count": null, "id": "ad297f19-998f-4485-aa2f-d67020058b7d", "metadata": {}, "outputs": [], "source": ["import os"]}, {"cell_type": "code", "execution_count": null, "id": "152ced37-9a42-47be-9a39-4218521f5e72", "metadata": {}, "outputs": [], "source": ["os.environ[\"REPLICATE_API_TOKEN\"] = \"<your API key>\""]}, {"cell_type": "markdown", "id": "deb2fb16-0a6a-4455-adb6-d0e7a29cfbbf", "metadata": {}, "source": ["## 基本用法\n"]}, {"cell_type": "markdown", "id": "6d7f254f-998b-4929-b887-fcdb34a85a2b", "metadata": {}, "source": ["我们展示了“vicuna-13b”模型，您可以直接在以下网址进行交互：https://replicate.com/replicate/vicuna-13b\n"]}, {"cell_type": "code", "execution_count": null, "id": "91210342-4494-4db6-b91a-59a552f0755b", "metadata": {}, "outputs": [], "source": ["from llama_index.llms.replicate import Replicate\n", "\n", "llm = Replicate(\n", "    model=\"replicate/vicuna-13b:6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b\"\n", ")"]}, {"cell_type": "markdown", "id": "b67c6a29-d407-4f00-9ef0-3a8116976ae9", "metadata": {}, "source": ["#### 使用提示调用`complete`\n"]}, {"cell_type": "code", "execution_count": null, "id": "d61b10bb-e911-47fb-8e84-19828cf224be", "metadata": {}, "outputs": [], "source": ["resp = llm.complete(\"Who is Paul Graham?\")"]}, {"cell_type": "code", "execution_count": null, "id": "3bd14f4e-c245-4384-a471-97e4ddfcb40e", "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["PaulGraham is a British physicist, mathematician, and computer scientist. He is best known for his work on the foundations of quantum mechanics and his contributions to the development of the field of quantum computing.\n", "\n", "Graham was born on August 15, 1957, in Cambridge, England. He received his undergraduate degree in mathematics from the University of Cambridge in 1979 and later earned his Ph.D. in theoretical physics from the University of California, Berkeley in 1984.\n", "\n", "Throughout his career, Graham has made significant contributions to the field of quantum mechanics. He has published a number of influential papers on the subject, including \"Quantum mechanics at 1/2 price,\" \"The holonomy of quantum mechanics,\" and \"Quantum mechanics in the presence of bounded self-adjoint operators.\"\n", "\n", "Graham has also been a key figure in the development of quantum computing. He is a co-founder of the quantum computing company, QxBranch, and has played a leading role in efforts to develop practical quantum algorithms and build large-scale quantum computers.\n", "\n", "In addition\n"]}], "source": ["print(resp)"]}, {"cell_type": "markdown", "id": "3ba9503c-b440-43c6-a50c-676c79993813", "metadata": {}, "source": ["#### 使用消息列表调用`chat`\n"]}, {"cell_type": "code", "execution_count": null, "id": "ee8a4a55-5680-4dc6-a44c-fc8ad7892f80", "metadata": {}, "outputs": [], "source": ["from llama_index.core.llms import ChatMessage\n", "\n", "messages = [\n", "    ChatMessage(\n", "        role=\"system\", content=\"You are a pirate with a colorful personality\"\n", "    ),\n", "    ChatMessage(role=\"user\", content=\"What is your name\"),\n", "]\n", "resp = llm.chat(messages)"]}, {"cell_type": "code", "execution_count": null, "id": "2a9bfe53-d15b-4e75-9d91-8c5d024f4eda", "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["assistant: ​\n"]}], "source": ["print(resp)"]}, {"cell_type": "markdown", "id": "25ad1b00-28fc-4bcd-96c4-d5b35605721a", "metadata": {}, "source": ["### 流式处理\n"]}, {"cell_type": "markdown", "id": "13c641fa-345a-4dce-87c5-ab1f6dcf4757", "metadata": {}, "source": ["使用 `stream_complete` 终端点\n"]}, {"cell_type": "code", "execution_count": null, "id": "06da1ef1-2f6b-497c-847b-62dd2df11491", "metadata": {}, "outputs": [], "source": ["response = llm.stream_complete(\"Who is Paul Graham?\")"]}, {"cell_type": "code", "execution_count": null, "id": "1b851def-5160-46e5-a30c-5a3ef2356b79", "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["PaulGraham is a British philosopher, cognitive scientist, and entrepreneur. He is best known for his work on the philosophy of the mind and consciousness, as well as his contributions to the development of the field of Artificial Intelligence (AI).\n", "\n", "Graham was born in London in 1938 and received his education at the University of Cambridge, where he studied philosophy and the natural sciences. After completing his studies, he went on to hold academic appointments at several prestigious universities, including the University of Oxford and the University of California, Berkeley.\n", "\n", "Throughout his career, Graham has been a prolific writer and thinker, publishing numerous articles and books on a wide range of topics, including the philosophy of mind, consciousness, AI, and the relationship between science and religion. He has also been involved in the development of several successful technology startups, including Viaweb (which was later acquired by Yahoo!) and Palantir Technologies.\n", "\n", "Despite his many achievements, Graham is perhaps best known for his contributions to the philosophy of the mind and consciousness. In particular, his work on the concept of"]}], "source": ["for r in response:\n", "    print(r.delta, end=\"\")"]}, {"cell_type": "markdown", "id": "ca52051d-6b28-49d7-98f5-82e266a1c7a6", "metadata": {}, "source": ["使用 `stream_chat` 端点\n"]}, {"cell_type": "code", "execution_count": null, "id": "fe553190-52a9-436d-84ae-4dd99a1808f4", "metadata": {}, "outputs": [], "source": ["from llama_index.core.llms import ChatMessage\n", "\n", "messages = [\n", "    ChatMessage(\n", "        role=\"system\", content=\"You are a pirate with a colorful personality\"\n", "    ),\n", "    ChatMessage(role=\"user\", content=\"What is your name\"),\n", "]\n", "resp = llm.stream_chat(messages)"]}, {"cell_type": "code", "execution_count": null, "id": "154c503c-f893-4b6b-8a65-a9a27b636046", "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["​"]}], "source": ["for r in resp:\n", "    print(r.delta, end=\"\")"]}, {"cell_type": "markdown", "id": "f317bf00-6296-4f20-9baf-321880753952", "metadata": {}, "source": ["## 配置模型\n"]}, {"cell_type": "code", "execution_count": null, "id": "0fec5774-3139-4258-869d-fb5b6f20ba73", "metadata": {}, "outputs": [], "source": ["from llama_index.llms.replicate import Replicate\n", "\n", "llm = Replicate(\n", "    model=\"replicate/vicuna-13b:6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b\",\n", "    temperature=0.9,\n", "    max_tokens=32,\n", ")"]}, {"cell_type": "code", "execution_count": null, "id": "92f24aae-ee9f-48ca-925b-949f9e72348f", "metadata": {}, "outputs": [], "source": ["resp = llm.complete(\"Who is Paul Graham?\")"]}, {"cell_type": "code", "execution_count": null, "id": "d89ee032-ffc4-45bc-b0f4-4dde8c97462b", "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["PaulGraham is an influential computer scientist, venture capitalist, and essayist. He is best known as\n"]}], "source": ["print(resp)"]}], "metadata": {"kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3"}}, "nbformat": 4, "nbformat_minor": 5}