{"cells":[{"cell_type":"markdown","id":"ba5f8741","metadata":{},"source":["# 使用插件检索的自定义代理\n","\n","该笔记本结合了两个概念，以构建一个可以与AI插件交互的自定义代理：\n","\n","1. [具有工具检索的自定义代理](/docs/modules/agents/how_to/custom_agent_with_tool_retrieval.html)：介绍了检索多个工具的概念，这在尝试使用任意数量的插件时非常有用。\n","2. [自然语言API链](/docs/use_cases/apis/openapi.html)：这在OpenAPI端点周围创建了自然语言包装器。这很有用，因为（1）插件在幕后使用OpenAPI端点，（2）将它们包装在NLAChain中使路由代理更容易调用。\n","\n","该笔记本引入的新颖想法是使用检索来选择不是显式工具，而是要使用的OpenAPI规范集。然后我们可以从这些OpenAPI规范生成工具。这种用例是在尝试让代理使用插件时。选择插件然后选择端点可能更有效，而不是直接选择端点。这是因为插件可能包含更有用的选择信息。"]},{"cell_type":"markdown","id":"fea4812c","metadata":{},"source":["## 设置环境\n","\n","进行必要的导入等操作。"]},{"cell_type":"code","execution_count":1,"id":"9af9734e","metadata":{},"outputs":[],"source":["# 导入所需的模块\n","import re\n","from typing import Union\n","\n","# 导入自定义模块\n","from langchain.agents import (\n","    AgentExecutor,\n","    AgentOutputParser,\n","    LLMSingleActionAgent,\n",")\n","from langchain.chains import LLMChain\n","from langchain.prompts import StringPromptTemplate\n","from langchain_community.agent_toolkits import NLAToolkit\n","from langchain_community.tools.plugin import AIPlugin\n","from langchain_core.agents import AgentAction, AgentFinish\n","from langchain_openai import OpenAI"]},{"cell_type":"markdown","id":"2f91d8b4","metadata":{},"source":["## 设置LLM"]},{"cell_type":"code","execution_count":2,"id":"a1a3b59c","metadata":{},"outputs":[],"source":["# 创建OpenAI对象，并设置温度参数为0\n","llm = OpenAI(temperature=0)"]},{"cell_type":"markdown","id":"6df0253f","metadata":{},"source":["## 设置插件\n","\n","加载并索引插件"]},{"cell_type":"code","execution_count":3,"id":"becda2a1","metadata":{},"outputs":[],"source":["# 定义一个包含多个URL的列表\n","urls = [\n","    \"https://datasette.io/.well-known/ai-plugin.json\",\n","    \"https://api.speak.com/.well-known/ai-plugin.json\",\n","    \"https://www.wolframalpha.com/.well-known/ai-plugin.json\",\n","    \"https://www.zapier.com/.well-known/ai-plugin.json\",\n","    \"https://www.klarna.com/.well-known/ai-plugin.json\",\n","    \"https://www.joinmilo.com/.well-known/ai-plugin.json\",\n","    \"https://slack.com/.well-known/ai-plugin.json\",\n","    \"https://schooldigger.com/.well-known/ai-plugin.json\",\n","]\n","\n","# 使用列表推导式从URL列表中创建AIPlugin对象的列表\n","AI_PLUGINS = [AIPlugin.from_url(url) for url in urls]"]},{"cell_type":"markdown","id":"17362717","metadata":{},"source":["## 工具检索器\n","\n","我们将使用一个向量存储器为每个工具描述创建嵌入。然后，对于传入的查询，我们可以为该查询创建嵌入，并对相关工具进行相似性搜索。"]},{"cell_type":"code","execution_count":4,"id":"77c4be4b","metadata":{},"outputs":[],"source":["从langchain_community.vectorstores导入FAISS模块\n","从langchain_core.documents导入Document模块\n","从langchain_openai导入OpenAIEmbeddings模块"]},{"cell_type":"code","execution_count":5,"id":"9092a158","metadata":{},"outputs":[{"name":"stderr","output_type":"stream","text":["Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n","Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n","Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n","Attempting to load an OpenAPI 3.0.2 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n","Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n","Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n","Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n","Attempting to load an OpenAPI 3.0.1 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n","Attempting to load a Swagger 2.0 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n"]}],"source":["# 导入OpenAIEmbeddings类\n","embeddings = OpenAIEmbeddings()\n","# 创建包含插件描述和元数据的文档列表\n","docs = [\n","    Document(\n","        page_content=plugin.description_for_model,\n","        metadata={\"plugin_name\": plugin.name_for_model},\n","    )\n","    for plugin in AI_PLUGINS\n","]\n","# 使用文档和嵌入向量创建向量存储\n","vector_store = FAISS.from_documents(docs, embeddings)\n","# 创建包含插件名称和NLAToolkit对象的字典\n","toolkits_dict = {\n","    plugin.name_for_model: NLAToolkit.from_llm_and_ai_plugin(llm, plugin)\n","    for plugin in AI_PLUGINS\n","}"]},{"cell_type":"code","execution_count":6,"id":"735a7566","metadata":{},"outputs":[],"source":["# 创建一个 retriever 对象，用于检索向量存储中的数据\n","retriever = vector_store.as_retriever()\n","\n","def get_tools(query):\n","    # 获取包含要使用的插件的文档\n","    docs = retriever.invoke(query)\n","    # 获取每个插件对应的工具包\n","    tool_kits = [toolkits_dict[d.metadata[\"plugin_name\"]] for d in docs]\n","    # 获取工具：每个端点对应一个单独的 NLAChain\n","    tools = []\n","    for tk in tool_kits:\n","        tools.extend(tk.nla_tools)\n","    return tools"]},{"cell_type":"markdown","id":"7699afd7","metadata":{},"source":["我们现在可以测试这个检索器，看看它是否能够工作。"]},{"cell_type":"code","execution_count":7,"id":"425f2886","metadata":{},"outputs":[{"data":{"text/plain":["['Milo.askMilo',\n"," 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.search_all_actions',\n"," 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.preview_a_zap',\n"," 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.get_configuration_link',\n"," 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.list_exposed_actions',\n"," 'SchoolDigger_API_V2.0.Autocomplete_GetSchools',\n"," 'SchoolDigger_API_V2.0.Districts_GetAllDistricts2',\n"," 'SchoolDigger_API_V2.0.Districts_GetDistrict2',\n"," 'SchoolDigger_API_V2.0.Rankings_GetSchoolRank2',\n"," 'SchoolDigger_API_V2.0.Rankings_GetRank_District',\n"," 'SchoolDigger_API_V2.0.Schools_GetAllSchools20',\n"," 'SchoolDigger_API_V2.0.Schools_GetSchool20',\n"," 'Speak.translate',\n"," 'Speak.explainPhrase',\n"," 'Speak.explainTask']"]},"execution_count":7,"metadata":{},"output_type":"execute_result"}],"source":["# 调用get_tools函数，并传入参数\"What could I do today with my kiddo\"，返回一个工具列表\n","tools = get_tools(\"What could I do today with my kiddo\")\n","\n","# 使用列表推导式，遍历工具列表，并返回每个工具的名称\n","[t.name for t in tools]"]},{"cell_type":"code","execution_count":8,"id":"3aa88768","metadata":{},"outputs":[{"data":{"text/plain":["['Open_AI_Klarna_product_Api.productsUsingGET',\n"," 'Milo.askMilo',\n"," 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.search_all_actions',\n"," 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.preview_a_zap',\n"," 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.get_configuration_link',\n"," 'Zapier_Natural_Language_Actions_(NLA)_API_(Dynamic)_-_Beta.list_exposed_actions',\n"," 'SchoolDigger_API_V2.0.Autocomplete_GetSchools',\n"," 'SchoolDigger_API_V2.0.Districts_GetAllDistricts2',\n"," 'SchoolDigger_API_V2.0.Districts_GetDistrict2',\n"," 'SchoolDigger_API_V2.0.Rankings_GetSchoolRank2',\n"," 'SchoolDigger_API_V2.0.Rankings_GetRank_District',\n"," 'SchoolDigger_API_V2.0.Schools_GetAllSchools20',\n"," 'SchoolDigger_API_V2.0.Schools_GetSchool20']"]},"execution_count":8,"metadata":{},"output_type":"execute_result"}],"source":["# 调用 get_tools 函数，传入参数 \"what shirts can i buy?\"，返回工具列表\n","tools = get_tools(\"what shirts can i buy?\") \n","\n","# 遍历工具列表，获取每个工具的名称，并存储在列表中\n","[t.name for t in tools]"]},{"cell_type":"markdown","id":"2e7a075c","metadata":{},"source":["## 提示模板\n","\n","提示模板非常标准，因为我们实际上并没有在实际提示模板中改变太多逻辑，而是改变了检索的方式。"]},{"cell_type":"code","execution_count":9,"id":"339b1bb8","metadata":{},"outputs":[],"source":["# 设置基本模板\n","template = \"\"\"以海盗的口吻回答以下问题，尽力而为。你可以使用以下工具：\n","\n","{tools}\n","\n","使用以下格式：\n","\n","问题：你必须回答的输入问题\n","思考：你应该始终考虑该怎么做\n","行动：要采取的行动，应该是[{tool_names}]之一\n","行动输入：行动的输入\n","观察：行动的结果\n","...（这个思考/行动/行动输入/观察可以重复N次）\n","思考：我现在知道最终答案了\n","最终答案：原始输入问题的最终答案\n","\n","开始！记住在给出最终答案时要以海盗的口吻说话。使用大量的\"Arg\"\n","\n","问题：{input}\n","{agent_scratchpad}\"\"\""]},{"cell_type":"markdown","id":"1583acdc","metadata":{},"source":["自定义提示模板现在具有一个名为tools_getter的概念，我们在输入上调用它来选择要使用的工具。"]},{"cell_type":"code","execution_count":10,"id":"fd969d31","metadata":{},"outputs":[],"source":["from typing import Callable\n","\n","\n","# 设置一个提示模板\n","class CustomPromptTemplate(StringPromptTemplate):\n","    # 要使用的模板\n","    template: str\n","    ############## 新增 ######################\n","    # 可用工具的列表\n","    tools_getter: Callable\n","\n","    def format(self, **kwargs) -> str:\n","        # 获取中间步骤（AgentAction，Observation元组）\n","        # 以特定方式格式化它们\n","        intermediate_steps = kwargs.pop(\"intermediate_steps\")\n","        thoughts = \"\"\n","        for action, observation in intermediate_steps:\n","            thoughts += action.log\n","            thoughts += f\"\\nObservation: {observation}\\nThought: \"\n","        # 将agent_scratchpad变量设置为该值\n","        kwargs[\"agent_scratchpad\"] = thoughts\n","        ############## 新增 ######################\n","        tools = self.tools_getter(kwargs[\"input\"])\n","        # 从提供的工具列表创建一个tools变量\n","        kwargs[\"tools\"] = \"\\n\".join(\n","            [f\"{tool.name}: {tool.description}\" for tool in tools]\n","        )\n","        # 为提供的工具创建一个工具名称列表\n","        kwargs[\"tool_names\"] = \", \".join([tool.name for tool in tools])\n","        return self.template.format(**kwargs)"]},{"cell_type":"code","execution_count":11,"id":"798ef9fb","metadata":{},"outputs":[],"source":["prompt = CustomPromptTemplate(\n","    template=template,  # 使用给定的模板创建自定义提示\n","    tools_getter=get_tools,  # 使用get_tools函数获取工具列表\n","    # 以下变量被动态生成，因此在这里省略：agent_scratchpad、tools和tool_names\n","    # intermediate_steps变量是必需的，因此在这里包含\n","    input_variables=[\"input\", \"intermediate_steps\"],  # 定义输入变量列表\n",")"]},{"cell_type":"markdown","id":"ef3a1af3","metadata":{},"source":["## 输出解析器\n","\n","输出解析器与之前的笔记本保持不变，因为我们不会对输出格式进行任何更改。"]},{"cell_type":"code","execution_count":12,"id":"7c6fe0d3","metadata":{},"outputs":[],"source":["class CustomOutputParser(AgentOutputParser):\n","    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:\n","        # 检查代理是否应该结束\n","        if \"Final Answer:\" in llm_output:\n","            return AgentFinish(\n","                # 返回值通常是一个只有一个 `output` 键的字典\n","                # 目前不建议尝试其他操作 :)\n","                return_values={\"output\": llm_output.split(\"Final Answer:\")[-1].strip()},\n","                log=llm_output,\n","            )\n","        # 解析出动作和动作输入\n","        regex = r\"Action\\s*\\d*\\s*:(.*?)\\nAction\\s*\\d*\\s*Input\\s*\\d*\\s*:[\\s]*(.*)\"\n","        match = re.search(regex, llm_output, re.DOTALL)\n","        if not match:\n","            raise ValueError(f\"无法解析LLM输出: `{llm_output}`\")\n","        action = match.group(1).strip()\n","        action_input = match.group(2)\n","        # 返回动作和动作输入\n","        return AgentAction(\n","            tool=action, tool_input=action_input.strip(\" \").strip('\"'), log=llm_output\n","        )"]},{"cell_type":"code","execution_count":13,"id":"d278706a","metadata":{},"outputs":[],"source":["output_parser = CustomOutputParser()  # 创建一个名为output_parser的对象，使用CustomOutputParser类进行初始化"]},{"cell_type":"markdown","id":"170587b1","metadata":{},"source":["## 设置LLM、停止序列和代理\n","\n","与之前的笔记本相同"]},{"cell_type":"code","execution_count":14,"id":"f9d4c374","metadata":{},"outputs":[],"source":["# 创建一个OpenAI对象，设置温度为0\n","llm = OpenAI(temperature=0)"]},{"cell_type":"code","execution_count":15,"id":"9b1cc2a2","metadata":{},"outputs":[],"source":["# 创建一个LLM链，包括LLM模型和一个提示\n","llm_chain = LLMChain(llm=llm, prompt=prompt)"]},{"cell_type":"code","execution_count":16,"id":"e4f5092f","metadata":{},"outputs":[],"source":["# 创建一个列表，包含了所有工具的名称\n","tool_names = [tool.name for tool in tools]\n","\n","# 创建一个LLMSingleActionAgent对象\n","# 参数说明：\n","# llm_chain: LLM链\n","# output_parser: 输出解析器\n","# stop: 停止标志，当遇到\"\\nObservation:\"时停止\n","# allowed_tools: 允许使用的工具名称列表\n","agent = LLMSingleActionAgent(\n","    llm_chain=llm_chain,\n","    output_parser=output_parser,\n","    stop=[\"\\nObservation:\"],\n","    allowed_tools=tool_names,\n",")"]},{"cell_type":"markdown","id":"aa8a5326","metadata":{},"source":["## 使用代理\n","\n","现在我们可以使用它了！"]},{"cell_type":"code","execution_count":17,"id":"490604e9","metadata":{},"outputs":[],"source":["# 导入AgentExecutor类\n","from rasa.core.agent import AgentExecutor\n","\n","# 创建AgentExecutor对象，并传入agent和tools参数，verbose参数设置为True\n","agent_executor = AgentExecutor.from_agent_and_tools(\n","    agent=agent, tools=tools, verbose=True\n",")"]},{"cell_type":"code","execution_count":18,"id":"653b1617","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["\n","\n","\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n","\u001b[32;1m\u001b[1;3mThought: I need to find a product API\n","Action: Open_AI_Klarna_product_Api.productsUsingGET\n","Action Input: shirts\u001b[0m\n","\n","Observation:\u001b[36;1m\u001b[1;3mI found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.\u001b[0m\u001b[32;1m\u001b[1;3m I now know what shirts I can buy\n","Final Answer: Arg, I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.\u001b[0m\n","\n","\u001b[1m> Finished chain.\u001b[0m\n"]},{"data":{"text/plain":["'Arg, I found 10 shirts from the API response. They range in price from $9.99 to $450.00 and come in a variety of materials, colors, and patterns.'"]},"execution_count":18,"metadata":{},"output_type":"execute_result"}],"source":["# 运行agent_executor并传入参数\"what shirts can i buy?\"\n","agent_executor.run(\"我可以买什么样的衬衫？\")"]},{"cell_type":"code","execution_count":null,"id":"2481ee76","metadata":{},"outputs":[],"source":[]}],"metadata":{"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.3"},"vscode":{"interpreter":{"hash":"18784188d7ecd866c0586ac068b02361a6896dc3a29b64f5cc957f09c590acef"}}},"nbformat":4,"nbformat_minor":5}