{"cells": [{"cell_type": "markdown", "id": "4ec7cd6e", "metadata": {}, "source": ["<a href=\"https://colab.research.google.com/github/jerryjliu/llama_index/blob/main/docs/docs/examples/llm/predibase.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"在 Colab 中打开\"/></a>\n"]}, {"cell_type": "markdown", "id": "bf9f19f3", "metadata": {}, "source": ["# Predibase\n", "\n", "这个笔记本展示了如何在Llamaindex中使用Predibase托管的LLM。您可以将[Predibase](https://predibase.com)添加到现有的Llamaindex工作流中，以便：\n", "1. 部署和查询预训练或自定义的开源LLM，无需麻烦\n", "2. 将端到端的检索增强生成（RAG）系统实现运营化\n", "3. 仅需几行代码即可对自己的LLM进行微调\n", "\n", "## 入门指南\n", "1. 在[这里](https://predibase.com/free-trial)免费注册Predibase账户\n", "2. 创建一个账户\n", "3. 转到设置 > 我的个人资料，并生成一个新的API令牌。\n"]}, {"cell_type": "code", "execution_count": null, "id": "72d6eb5b", "metadata": {}, "outputs": [], "source": ["%pip install llama-index-llms-predibase"]}, {"cell_type": "code", "execution_count": null, "id": "79a726c5", "metadata": {}, "outputs": [], "source": ["!pip install llama-index --quiet\n", "!pip install predibase --quiet\n", "!pip install sentence-transformers --quiet"]}, {"cell_type": "code", "execution_count": null, "id": "1c2b0d5d", "metadata": {}, "outputs": [], "source": ["import os\n", "\n", "os.environ[\"PREDIBASE_API_TOKEN\"] = \"{PREDIBASE_API_TOKEN}\"\n", "from llama_index.llms.predibase import PredibaseLLM"]}, {"cell_type": "markdown", "id": "9a602a2a", "metadata": {}, "source": ["## 流程 1：直接查询Predibase LLM\n", "\n", "在这个流程中，我们将直接查询Predibase LLM数据库，以获取所需的信息。\n"]}, {"cell_type": "code", "execution_count": null, "id": "4baffaa2", "metadata": {}, "outputs": [], "source": ["# Predibase托管的微调适配器示例\n", "llm = PredibaseLLM(\n", "    model_name=\"mistral-7b\",\n", "    predibase_sdk_version=None,  # 可选参数（如果省略，默认为最新的Predibase SDK版本）\n", "    adapter_id=\"e2e_nlg\",  # adapter_id是可选的\n", "    adapter_version=1,  # 可选参数（仅适用于Predibase）\n", "    temperature=0.3,\n", "    max_new_tokens=512,\n", ")\n", "# `model_name`参数是Predibase的“无服务器”base_model ID\n", "# （请参阅https://docs.predibase.com/user-guide/inference/models以获取目录）。\n", "# 您还可以选择指定托管在Predibase或HuggingFace上的微调适配器\n", "# 对于Predibase托管的适配器，您还必须指定adapter_version"]}, {"cell_type": "code", "execution_count": null, "id": "69713553", "metadata": {}, "outputs": [], "source": ["# 基于HuggingFace托管的微调适配器示例\n", "llm = PredibaseLLM(\n", "    model_name=\"mistral-7b\",\n", "    predibase_sdk_version=None,  # 可选参数（如果省略，默认为最新的Predibase SDK版本）\n", "    adapter_id=\"predibase/e2e_nlg\",  # adapter_id是可选的\n", "    temperature=0.3,\n", "    max_new_tokens=512,\n", ")\n", "# `model_name`参数是Predibase的“无服务器”base_model ID\n", "# （参见https://docs.predibase.com/user-guide/inference/models获取目录）。\n", "# 您还可以选择指定托管在Predibase或HuggingFace上的微调适配器\n", "# 对于Predibase托管的适配器，您还可以指定adapter_version（如果省略，假定为最新版本）"]}, {"cell_type": "code", "execution_count": null, "id": "e7039a65", "metadata": {}, "outputs": [], "source": ["result = llm.complete(\"Can you recommend me a nice dry white wine?\")\n", "print(result)"]}, {"cell_type": "markdown", "id": "1112e828", "metadata": {}, "source": ["## 流程2：检索增强生成（RAG）与Predibase LLM\n", "\n", "RAG是一种结合了检索和生成的模型，可以用于生成与给定查询相关的文本。Predibase LLM是一种基于语言的模型，用于生成与给定主题相关的文本。在这个流程中，我们将结合使用RAG和Predibase LLM，以实现检索增强生成的效果。\n"]}, {"cell_type": "code", "execution_count": null, "id": "cacff36a", "metadata": {}, "outputs": [], "source": ["from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n", "from llama_index.core.embeddings import resolve_embed_model\n", "from llama_index.core.node_parser import SentenceSplitter"]}, {"attachments": {}, "cell_type": "markdown", "id": "c8f6fef1", "metadata": {}, "source": ["#### 下载数据\n"]}, {"cell_type": "code", "execution_count": null, "id": "65930e7e", "metadata": {}, "outputs": [], "source": ["!mkdir -p 'data/paul_graham/'\n", "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'"]}, {"cell_type": "markdown", "id": "1edd41d1", "metadata": {}, "source": ["### 加载文档\n"]}, {"cell_type": "code", "execution_count": null, "id": "c5941151", "metadata": {}, "outputs": [], "source": ["documents = SimpleDirectoryReader(\"./data/paul_graham/\").load_data()"]}, {"cell_type": "markdown", "id": "7df4407f", "metadata": {}, "source": ["### 配置Predibase LLM\n"]}, {"cell_type": "code", "execution_count": null, "id": "3f67e975-3cb5-4ddc-98e8-eae7892315ca", "metadata": {}, "outputs": [], "source": ["# Predibase托管的经过微调的适配器\n", "llm = PredibaseLLM(\n", "    model_name=\"mistral-7b\",\n", "    predibase_sdk_version=None,  # 可选参数（如果省略，则默认为最新的Predibase SDK版本）\n", "    adapter_id=\"e2e_nlg\",  # adapter_id是可选的\n", "    temperature=0.3,\n", "    context_window=1024,\n", ")"]}, {"cell_type": "code", "execution_count": null, "id": "4a44defc", "metadata": {}, "outputs": [], "source": ["# HuggingFace托管的微调适配器\n", "llm = PredibaseLLM(\n", "    model_name=\"mistral-7b\",\n", "    predibase_sdk_version=None,  # 可选参数（如果省略，默认为最新的Predibase SDK版本）\n", "    adapter_id=\"predibase/e2e_nlg\",  # adapter_id是可选的\n", "    temperature=0.3,\n", "    context_window=1024,\n", ")"]}, {"cell_type": "code", "execution_count": null, "id": "b3d527b7-5110-4d9c-97df-3926d3db0772", "metadata": {}, "outputs": [], "source": ["embed_model = resolve_embed_model(\"local:BAAI/bge-small-en-v1.5\")\n", "splitter = SentenceSplitter(chunk_size=1024)"]}, {"cell_type": "markdown", "id": "7a131a8e", "metadata": {}, "source": ["### 设置和查询索引\n"]}, {"cell_type": "code", "execution_count": null, "id": "c9b10269", "metadata": {}, "outputs": [], "source": ["index = VectorStoreIndex.from_documents(\n", "    documents, transformations=[splitter], embed_model=embed_model\n", ")\n", "query_engine = index.as_query_engine(llm=llm)\n", "response = query_engine.query(\"What did the author do growing up?\")"]}, {"cell_type": "code", "execution_count": null, "id": "ac73eb65", "metadata": {}, "outputs": [], "source": ["print(response)"]}], "metadata": {"kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3"}, "vscode": {"interpreter": {"hash": "5ae9fa2777630f93d325d67fd0c37f7375ed1afcb20dd85f425eb8692a47ff3f"}}}, "nbformat": 4, "nbformat_minor": 5}