{"cells": [{"attachments": {}, "cell_type": "markdown", "id": "93e3f84f", "metadata": {}, "source": ["\"在Colab中打开\"/\n"]}, {"attachments": {}, "cell_type": "markdown", "id": "11b52622-f38a-4b3a-a916-c73619babb48", "metadata": {}, "source": ["# LLM重新排名演示（2021 Lyft 10-k）\n", "\n", "本教程展示了如何进行两阶段检索。使用基于嵌入的检索，设置一个较高的top-k值，以最大化召回率并获得大量候选项。然后，使用基于LLM的检索动态选择实际与查询相关的节点。\n"]}, {"cell_type": "code", "execution_count": null, "id": "e66e4c0c", "metadata": {}, "outputs": [], "source": ["%pip install llama-index-llms-openai"]}, {"cell_type": "code", "execution_count": null, "id": "91b61e96-864c-4ed2-80d6-0ebfdbb57d5c", "metadata": {}, "outputs": [], "source": ["import nest_asyncio\n", "\n", "nest_asyncio.apply()"]}, {"cell_type": "code", "execution_count": null, "id": "f630346f-fedd-40f2-8fb3-e052e219a873", "metadata": {}, "outputs": [], "source": ["import logging\n", "import sys\n", "\n", "logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n", "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n", "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n", "from llama_index.core.postprocessor import LLMRerank\n", "\n", "from llama_index.llms.openai import OpenAI\n", "from IPython.display import Markdown, display"]}, {"attachments": {}, "cell_type": "markdown", "id": "231c4418", "metadata": {}, "source": ["## 下载数据\n"]}, {"cell_type": "code", "execution_count": null, "id": "36f2294f", "metadata": {}, "outputs": [], "source": ["!mkdir -p 'data/10k/'\n", "!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10k/lyft_2021.pdf' -O 'data/10k/lyft_2021.pdf'"]}, {"attachments": {}, "cell_type": "markdown", "id": "8a8fdeb2-939c-49dd-b8e6-0139d53a4fb6", "metadata": {}, "source": ["## 加载数据，构建索引\n"]}, {"cell_type": "code", "execution_count": null, "id": "ee132733-b525-4aaf-81db-7eed19ded815", "metadata": {}, "outputs": [], "source": ["from llama_index.core import Settings\n", "\n", "# LLM (gpt-3.5-turbo)\n", "Settings.llm = OpenAI(温度=0, 模型=\"gpt-3.5-turbo\")\n", "\n", "Settings.chunk_overlap = 0\n", "Settings.chunk_size = 128"]}, {"cell_type": "code", "execution_count": null, "id": "88344ea2-540c-4b46-8a66-ed78870cb80a", "metadata": {}, "outputs": [], "source": ["# 加载文档\n", "documents = SimpleDirectoryReader(\n", " input_files=[\"./data/10k/lyft_2021.pdf\"]\n", ").load_data()"]}, {"cell_type": "code", "execution_count": null, "id": "d478f416-8e85-49ae-9817-e4d78122a120", "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n", "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n", "> [build_index_from_nodes] Total LLM token usage: 0 tokens\n", "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 226241 tokens\n", "> [build_index_from_nodes] Total embedding token usage: 226241 tokens\n", "> [build_index_from_nodes] Total embedding token usage: 226241 tokens\n"]}], "source": ["index = VectorStoreIndex.from_documents(\n", " documents,\n", ")"]}, {"attachments": {}, "cell_type": "markdown", "id": "7e3d5f23-dfcd-458d-a9d3-dd66de0ab054", "metadata": {}, "source": ["## 检索比较\n"]}, {"cell_type": "code", "execution_count": null, "id": "47480c6d-8914-4562-a789-fd53a99a7afb", "metadata": {}, "outputs": [{"name": "stderr", "output_type": "stream", "text": ["/var/folders/1r/c3h91d9s49xblwfvz79s78_c0000gn/T/ipykernel_58458/2502541873.py:8: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.\n", " pd.set_option('display.max_colwidth', -1)\n"]}], "source": ["from llama_index.core.retrievers import VectorIndexRetriever\n", "from llama_index.core import QueryBundle\n", "import pandas as pd\n", "from IPython.display import display, HTML\n", "from copy import deepcopy\n", "\n", "\n", "pd.set_option(\"display.max_colwidth\", -1)\n", "\n", "\n", "def get_retrieved_nodes(\n", " query_str, vector_top_k=10, reranker_top_n=3, with_reranker=False\n", "):\n", " query_bundle = QueryBundle(query_str)\n", " # 配置检索器\n", " retriever = VectorIndexRetriever(\n", " index=index,\n", " similarity_top_k=vector_top_k,\n", " )\n", " retrieved_nodes = retriever.retrieve(query_bundle)\n", "\n", " if with_reranker:\n", " # 配置重新排序器\n", " reranker = LLMRerank(\n", " choice_batch_size=5,\n", " top_n=reranker_top_n,\n", " )\n", " retrieved_nodes = reranker.postprocess_nodes(\n", " retrieved_nodes, query_bundle\n", " )\n", "\n", " return retrieved_nodes\n", "\n", "\n", "def pretty_print(df):\n", " return display(HTML(df.to_html().replace(\"\\\\n\", \"
\")))\n", "\n", "\n", "def visualize_retrieved_nodes(nodes) -> None:\n", " result_dicts = []\n", " for node in nodes:\n", " node = deepcopy(node)\n", " node.node.metadata = None\n", " node_text = node.node.get_text()\n", " node_text = node_text.replace(\"\\n\", \" \")\n", "\n", " result_dict = {\"Score\": node.score, \"Text\": node_text}\n", " result_dicts.append(result_dict)\n", "\n", " pretty_print(pd.DataFrame(result_dicts))"]}, {"cell_type": "code", "execution_count": null, "id": "b8bedc4f-444b-4233-9b72-728e3cfbe056", "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n", "> [retrieve] Total LLM token usage: 0 tokens\n", "> [retrieve] Total LLM token usage: 0 tokens\n", "> [retrieve] Total LLM token usage: 0 tokens\n", "> [retrieve] Total LLM token usage: 0 tokens\n", "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 11 tokens\n", "> [retrieve] Total embedding token usage: 11 tokens\n", "> [retrieve] Total embedding token usage: 11 tokens\n", "> [retrieve] Total embedding token usage: 11 tokens\n", "> [retrieve] Total embedding token usage: 11 tokens\n"]}], "source": ["new_nodes = get_retrieved_nodes(\n", " \"What is Lyft's response to COVID-19?\", vector_top_k=5, with_reranker=False\n", ")"]}, {"cell_type": "code", "execution_count": null, "id": "e85e656b-9377-4640-a10d-a6655afd82bd", "metadata": {}, "outputs": [{"data": {"text/html": ["\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ScoreText
00.863554Rentals. Further, COVID-19 has and may continue to negatively impact Lyft’s ability to conduct rental operationsthrough the Express Drive program and Lyft Rentals as a result of restrictions on travel, mandated closures, limited staffing availability, and other factors relatedto COVID-19. For example, in 2020, Lyft Rentals temporarily ceased operations, closing its rental locations, as a result of COVID-19. Further, while ExpressDrive rental periods
10.854175pandemic, including sales, marketing and costs relating to our efforts to mitigate the impact of the COVID-19 pandemic. Furthermore, we have expanded overtime to include more asset-intensive offerings such as our network of Light Vehicles, Flexdrive, Lyft Rentals and Lyft Auto Care. We are also expanding the supportavailable to drivers at our Driver Hubs, our driver-centric service centers and community spaces, Driver Centers, our vehicle service centers, Mobile Services,
20.852866requested to quarantine by a medical professional, which it continues to do at this time. Further, Lyft Rentals and Flexdrive have facedsignificantly higher cos ts in transporting, repossessing, cleaning, and17
30.847151the transport ation needs of customers, employees and other constituents.• Grow Active Riders. We see opportunities to continue to recoup and grow our rider base amid the continuing COVID-19 pandemic. We may make incrementalinvestments in our brand and in growth marketing to maintain and drive increasing consumer preference for Lyft. We may also offer discounts for first-time ridersto try Lyft or provide incentives to existing riders to encourage increased ride frequency. We
40.841177day one, we have worked continuousl y to enhance the safety of our platform and the ridesharing industry by developing innovative products, policiesand processes. Business Lyft is evolving how businesses large and small take care of their people’s transportation needs across sectors including corporate, healthcare, auto, education andgovernment. Our comprehensive set of solutions allows clients to design, manage and pay for ground
"], "text/plain": [""]}, "metadata": {}, "output_type": "display_data"}], "source": ["visualize_retrieved_nodes(new_nodes)"]}, {"cell_type": "code", "execution_count": null, "id": "8ba150e2-c4e7-4404-b8e1-1603c2b346d3", "metadata": {}, "outputs": [{"name": "stdout", "output_type": "stream", "text": ["INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens\n", "> [retrieve] Total LLM token usage: 0 tokens\n", "> [retrieve] Total LLM token usage: 0 tokens\n", "> [retrieve] Total LLM token usage: 0 tokens\n", "> [retrieve] Total LLM token usage: 0 tokens\n", "INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 11 tokens\n", "> [retrieve] Total embedding token usage: 11 tokens\n", "> [retrieve] Total embedding token usage: 11 tokens\n", "> [retrieve] Total embedding token usage: 11 tokens\n", "> [retrieve] Total embedding token usage: 11 tokens\n"]}], "source": ["new_nodes = get_retrieved_nodes(\n", " \"What is Lyft's response to COVID-19?\",\n", " vector_top_k=20,\n", " reranker_top_n=5,\n", " with_reranker=True,\n", ")"]}, {"cell_type": "code", "execution_count": null, "id": "7541606f-6424-470b-987c-a986ac0a7cf8", "metadata": {}, "outputs": [{"data": {"text/html": ["\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ScoreText
010.0inunrestricted cash and cash equivalents and short-term investments as of December 31, 2021, we believe we have sufficient liquidity to continue business operations and totake action we determine to be in the best interests of our employees, stockholders, stakeholders and of drivers and riders on the Lyft Platform. For more information onrisks associated with the COVID-19 pandem ic, see the section titled “Risk Factors” in Item 1A of Part I.Recent Developments Transaction
110.0COVID-19, may continue to develop or persist over time and further contribute to thisadverse effect. • Changes in driver behavior during the COVID-19 pandemic have led to reduced levels of driver availability on our platform relative to rider demand in certainmarkets. This imbalance fluctuates for various reasons, and to the extent that driver availability is limited, our service levels have been and may be negativelyimpacted and we have increased prices or provided additional incentives and may need to continue to do so, which
210.0estimated.In response to the COVID-19 pandemic, we have adopted multiple measures, including, but not limited, to establishing new health and safety requirements forridesharing and updating workplace policies. We also made adjustments to our expenses and cash flow to correlate with declines in revenues including headcountreductions in 2020. 56
310.0opportunities for drivers on our platform. Our business continues to be impacted by the COVID-19pandemic. Although we have seen some signs of demand improving, particularly compared to the demand levels at the start of the pandemic, demand levels continue to beaffected by the impact of variants and changes in case counts. The exact timing and pace of the recovery remain uncertain. The extent to which our operations will continueto be impacted by the pandemic will depend largely on future
410.0does not perceive ridesharing or our other offerings as beneficial, or chooses not to adopt them as a result of concerns regarding public health or safety, affordability or forother reasons, whether as a result of incidents on our platform or on our competitors’ platforms, the COVID-19 pandemic, or otherwise, then the market for our offeringsmay not further develop, may develop more slowly than we expect or may not achieve the growth potential we expect. Additionally,
"], "text/plain": [""]}, "metadata": {}, "output_type": "display_data"}], "source": ["visualize_retrieved_nodes(new_nodes)"]}, {"cell_type": "code", "execution_count": null, "id": "eb88c0bb-4d3f-4426-b2be-66b0d5635abb", "metadata": {}, "outputs": [], "source": ["new_nodes = get_retrieved_nodes(\n", " \"What initiatives are the company focusing on independently of COVID-19?\",\n", " vector_top_k=5,\n", " with_reranker=False,\n", ")"]}, {"cell_type": "code", "execution_count": null, "id": "3c4b3962-2873-40b3-9f50-58cf7685454a", "metadata": {}, "outputs": [{"data": {"text/html": ["\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ScoreText
00.819209businesses to contain the pandemic or respond to its impact and altered consumer behavior, amongother things. The Company has adopted a number of measures in response to the COVID-19 pandemic including, but not limited to, establishing new health and safetyrequirements for ridesharing and updating workplace policies. The Company also made adjustments to its expenses and cash flow to correlate with declines in revenuesincluding headcount reductions in 2020. Refer to Note 17 “Restructuring” to the
10.813341business;• manage our platform and our business assets and expenses in light of the COVID-19 pandemic and related public health measures issued by various jurisdictions,including travel bans, travel restrictions and shelter-in-place orders, as well as maintain demand for and confidence in the safety of our platform during andfollowing the COVID-19 pandemic; • plan for and manage capital
20.809412pandemic, including sales, marketing and costs relating to our efforts to mitigate the impact of the COVID-19 pandemic. Furthermore, we have expanded overtime to include more asset-intensive offerings such as our network of Light Vehicles, Flexdrive, Lyft Rentals and Lyft Auto Care. We are also expanding the supportavailable to drivers at our Driver Hubs, our driver-centric service centers and community spaces, Driver Centers, our vehicle service centers, Mobile Services,
30.809215COVID-19 pandemic in March 2020. We have adoptedmultiple measures in response to the COVID-19 pandemic. We cannot be certain that these actions will mitigate some or all of the negative effects of the pandemic on ourbusiness. In light of the evolving and unpredictable effects of COVID-19, we are not currently in a position to forecast the expected impact of COVID-19 on our financialand operating results in fu ture periods.Revenue Recognition Revenue
40.808421estimated.In response to the COVID-19 pandemic, we have adopted multiple measures, including, but not limited, to establishing new health and safety requirements forridesharing and updating workplace policies. We also made adjustments to our expenses and cash flow to correlate with declines in revenues including headcountreductions in 2020. 56
"], "text/plain": [""]}, "metadata": {}, "output_type": "display_data"}], "source": ["visualize_retrieved_nodes(new_nodes)"]}, {"cell_type": "code", "execution_count": null, "id": "1dcc798a-3940-4426-8110-d9a3f7dc5a68", "metadata": {}, "outputs": [], "source": ["new_nodes = get_retrieved_nodes(\n", " \"What initiatives are the company focusing on independently of COVID-19?\",\n", " vector_top_k=40,\n", " reranker_top_n=5,\n", " with_reranker=True,\n", ")"]}, {"cell_type": "code", "execution_count": null, "id": "38bae391-7dca-4f8d-a90e-ada1beddcd2e", "metadata": {}, "outputs": [{"data": {"text/html": ["\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ScoreText
010.0remotely, as well as permanent return to workarrangements and workplac e strategies;• the inability to achieve adherence to our internal policies and core values, including our diversity, equity and inclusion practices and initiatives;• competitive pressures to move in directions that may divert us from our mission, vision and values;• the continued challenges of a rapidly-evolving industry;• the increasing need to develop expertise in new areas of business that
19.0platfor m and scaled user network.Notwithstanding the impact of COVID-19, we are continuing to invest in the future, both organically and through acquisitions of complementary businesses. Wealso continue to invest in the expansion of our network of Light Vehicles and Lyft Autonomous, which focuses on the deployment and scaling of third-party self-drivingtechnology on the Lyft network. Our strategy is to always be at the forefront of transportation innovation, and we believe that through these
29.0the transport ation needs of customers, employees and other constituents.• Grow Active Riders. We see opportunities to continue to recoup and grow our rider base amid the continuing COVID-19 pandemic. We may make incrementalinvestments in our brand and in growth marketing to maintain and drive increasing consumer preference for Lyft. We may also offer discounts for first-time ridersto try Lyft or provide incentives to existing riders to encourage increased ride frequency. We
38.0to grow our business and improve ourofferings, we will face challenges related to providing quality support services at scale. If we grow our international rider base and the number of international drivers onour platform, our support organization will face additional challenges, including those associated with delivering support in languages other than English. Furthermore, theCOVID-19 pandemic may impact our ability to provide effective and timely support, including as a result of a decrease in the availability of service providers and increasein
46.0pandemic and responsive measures;• natural disasters, economic downturns, public health crises or political crises;• general macroeconomic conditions;Operational factors • our limited operating history;• our financial performance and any inability to achieve or maintain profitability in the future;• competition in our industries;• the unpredictability of our results of operations;• uncertainty regarding the growth of the ridesharing and other markets;• our ability to attract and
"], "text/plain": [""]}, "metadata": {}, "output_type": "display_data"}], "source": ["visualize_retrieved_nodes(new_nodes)"]}], "metadata": {"kernelspec": {"display_name": "llama_index_v2", "language": "python", "name": "llama_index_v2"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3"}}, "nbformat": 4, "nbformat_minor": 5}