{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "u27oioiVYe8q",
   "metadata": {
    "id": "u27oioiVYe8q"
   },
   "source": [
    "# Having It Both Ways: Combining BM25 with AI Reranking"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rRHsoGTpXLhI",
   "metadata": {
    "id": "rRHsoGTpXLhI"
   },
   "source": [
    "You can find the original blog post under the following [link](https://jina.ai/news/having-it-both-ways-combining-bm25-with-ai-reranking)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9Fre11ChX84L",
   "metadata": {
    "id": "9Fre11ChX84L"
   },
   "source": [
    "### Upload files to Google Colab"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "czD_dUZiYIkb",
   "metadata": {
    "id": "czD_dUZiYIkb"
   },
   "source": [
    "Before we can access local files on Google Colab, we need to upload them to the Colab environment. Here are the steps to do so:\n",
    "    \n",
    "1. [Download the file `fashion_data.csv`](https://raw.githubusercontent.com/jina-ai/workshops/main/notebooks/embeddings/bm25/fashion_data.csv) to your local drive.\n",
    "2. Click on the “Files” tab on the left-side menu in Google Colab (Make sure it is the “Files tab” not the “File” Dropdown menu).\n",
    "3. Click on the “Upload to Session Storage” button and select the `fashion_data.csv` file you previously downloaded.\n",
    "4. Wait for the upload to complete.\n",
    "\n",
    "Once the `fashion_data.csv` file is uploaded, you can access it in the “Files” tab."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4_GuyXXxYmIa",
   "metadata": {
    "id": "4_GuyXXxYmIa"
   },
   "source": [
    "Install prerequisites:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19521118-7a1f-4744-8a25-c3376822f280",
   "metadata": {
    "id": "19521118-7a1f-4744-8a25-c3376822f280"
   },
   "outputs": [],
   "source": [
    "!pip install --q haystack-ai jina-haystack"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "orAUUDYZYzfG",
   "metadata": {
    "id": "orAUUDYZYzfG"
   },
   "source": [
    "Add the Jina API key as environment variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd7b5971-5267-487f-85d0-88b97ae8cd0c",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1310679,
     "status": "ok",
     "timestamp": 1713884793971,
     "user": {
      "displayName": "Francesco Kruk",
      "userId": "12659492613492773792"
     },
     "user_tz": -120
    },
    "id": "bd7b5971-5267-487f-85d0-88b97ae8cd0c",
    "outputId": "ce92f659-e0ef-4cd6-a0b8-fd1dfa60af78"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import getpass\n",
    "\n",
    "os.environ[\"JINA_API_KEY\"] = getpass.getpass()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "_tDBUC_8Ys2u",
   "metadata": {
    "id": "_tDBUC_8Ys2u"
   },
   "source": [
    "Define the query in form of the product category:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7c2cb7c-f8ae-4f93-a976-a98e62559845",
   "metadata": {
    "id": "f7c2cb7c-f8ae-4f93-a976-a98e62559845"
   },
   "outputs": [],
   "source": [
    "query = \"Nightwear for Women\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5vOLK2akY2Gz",
   "metadata": {
    "id": "5vOLK2akY2Gz"
   },
   "source": [
    "Transform the data into Documents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61ed774f-985a-4ba9-b674-62ba9322a8ba",
   "metadata": {
    "id": "61ed774f-985a-4ba9-b674-62ba9322a8ba"
   },
   "outputs": [],
   "source": [
    "import csv\n",
    "from haystack import Document\n",
    "\n",
    "documents = []\n",
    "with open(\"fashion_data.csv\") as f:\n",
    "    data = csv.reader(f, delimiter=\";\")\n",
    "    for row in data:\n",
    "      row_text = ''.join(row)\n",
    "      row_doc = Document(content=row_text, meta={\"prod_id\": row[0], \"prod_image\": row[1]})\n",
    "      documents.append(row_doc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "CRLmeOrxY_Hr",
   "metadata": {
    "id": "CRLmeOrxY_Hr"
   },
   "source": [
    "Create the query pipeline WITHOUT Jina Reranker to compare the results prior to the reranking:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a053845d-0e86-4bc3-bea9-aea5071dc1f3",
   "metadata": {
    "id": "a053845d-0e86-4bc3-bea9-aea5071dc1f3"
   },
   "outputs": [],
   "source": [
    "from haystack import Pipeline\n",
    "from haystack.document_stores.types import DuplicatePolicy\n",
    "from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
    "from haystack.components.retrievers.in_memory import InMemoryBM25Retriever\n",
    "\n",
    "document_store=InMemoryDocumentStore()\n",
    "document_store.write_documents(documents=documents, policy=DuplicatePolicy.OVERWRITE)\n",
    "\n",
    "retriever = InMemoryBM25Retriever(document_store=document_store)\n",
    "\n",
    "rag_pipeline = Pipeline()\n",
    "rag_pipeline.add_component(\"retriever\", retriever)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "XE5nQo4PZIyy",
   "metadata": {
    "id": "XE5nQo4PZIyy"
   },
   "source": [
    "Run the query pipeline WITHOUT Jina Reranker:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f66eb1c-8d14-4192-a810-4fa763406391",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000,
     "referenced_widgets": [
      "2a9652e7558a48d48eb2821ea49de1c5",
      "f219657745f341d7b03b812d990f2b10",
      "9fb25fb75bd54aab971304bbc6210a8f",
      "4b19c42c412b46e2bd819f305c6f076d",
      "c511e21371fa4d388fcfd9d9e4618621",
      "55756a04a1d94c63a69439b8195314a9",
      "4fe2fef4428e4105bc2a615ab4ae878d",
      "1eb03a5ec71c422f9e84b8f87e8cf87e",
      "d01408fff8a945bf9197238b58c5dd40",
      "d11e0276f5ec4169b534013d430a1446",
      "bf8db219aba845f2abd7a80b9dc9c898"
     ]
    },
    "executionInfo": {
     "elapsed": 8391,
     "status": "ok",
     "timestamp": 1713884806908,
     "user": {
      "displayName": "Francesco Kruk",
      "userId": "12659492613492773792"
     },
     "user_tz": -120
    },
    "id": "7f66eb1c-8d14-4192-a810-4fa763406391",
    "outputId": "d44dbb58-87e7-45c8-ba0f-544c099c9a7f"
   },
   "outputs": [],
   "source": [
    "result = rag_pipeline.run(\n",
    "            {\n",
    "                \"retriever\": {\"query\": query, \"top_k\": 50},\n",
    "            }\n",
    "        )\n",
    "\n",
    "for doc in result[\"retriever\"][\"documents\"]:\n",
    "    print(\"Product ID:\", doc.meta[\"prod_id\"])\n",
    "    print(\"Product Image:\", doc.meta[\"prod_image\"])\n",
    "    print(\"Score:\", doc.score)\n",
    "    print(\"-\"*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "JX7udshKBoID",
   "metadata": {
    "id": "JX7udshKBoID"
   },
   "source": [
    "![image.png](./images/bm25-retrieved-results.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "QHXHJ3uAF0hl",
   "metadata": {
    "id": "QHXHJ3uAF0hl"
   },
   "source": [
    "*As we can see, although the results are related to the nightwear we asked for, the most relevant matches seem to get lost within the multitude of products retrieved by BM25. Concretely, this means that a user would mainly receive unrelated results at the top of the page which might not match their exact needs.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Teq-ZwCUZKI0",
   "metadata": {
    "id": "Teq-ZwCUZKI0"
   },
   "source": [
    "Create the query pipeline WITH Jina Reranker to compare the results after the reranking:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ca8155a-d035-4267-932a-e2dd581accb2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 53,
     "status": "ok",
     "timestamp": 1713884806908,
     "user": {
      "displayName": "Francesco Kruk",
      "userId": "12659492613492773792"
     },
     "user_tz": -120
    },
    "id": "9ca8155a-d035-4267-932a-e2dd581accb2",
    "outputId": "d0ee640a-0482-444c-a156-2ed9409d3153"
   },
   "outputs": [],
   "source": [
    "from haystack_integrations.components.rankers.jina import JinaRanker\n",
    "\n",
    "ranker_retriever = InMemoryBM25Retriever(document_store=document_store)\n",
    "\n",
    "ranker = JinaRanker()\n",
    "\n",
    "ranker_pipeline = Pipeline()\n",
    "ranker_pipeline.add_component(\"ranker_retriever\", ranker_retriever)\n",
    "ranker_pipeline.add_component(\"ranker\", ranker)\n",
    "\n",
    "ranker_pipeline.connect(\"ranker_retriever.documents\", \"ranker.documents\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "lk980IdQZQHC",
   "metadata": {
    "id": "lk980IdQZQHC"
   },
   "source": [
    "Run the query pipeline WITH Jina Reranker:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32117df4-d11f-4684-addd-b37a479cae04",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000,
     "referenced_widgets": [
      "1b58f96e6bf449948a8a8ba747f3655f",
      "38938aa4c9bc4965b00cf55231c3fe44",
      "3e1b3bf8208044edbb545d482c4179c7",
      "2fd9aa10863f4ce9aa7851aa28451c69",
      "dabc3892bc50408f926229dee474fbcb",
      "feaf0bcb4ce54950bbe027e97d29e7fe",
      "0bd040d1e5774fa9819bfda81c162a45",
      "5fe3ba5614264d43888c61b1e4a1e93c",
      "052c5babc9894ed598d1e6c685df6480",
      "f7170b0156324004bcc8eeff0ac8697a",
      "fc9abd42e5d24b538fb60e6c3d7233fb"
     ]
    },
    "executionInfo": {
     "elapsed": 4387,
     "status": "ok",
     "timestamp": 1713884811274,
     "user": {
      "displayName": "Francesco Kruk",
      "userId": "12659492613492773792"
     },
     "user_tz": -120
    },
    "id": "32117df4-d11f-4684-addd-b37a479cae04",
    "outputId": "bc7bdfbb-f8d9-4d8f-b81f-704a50c08f6f"
   },
   "outputs": [],
   "source": [
    "result = ranker_pipeline.run(\n",
    "            {\n",
    "                \"ranker_retriever\": {\"query\": query, \"top_k\": 50},\n",
    "                \"ranker\": {\"query\": query, \"top_k\": 10},\n",
    "            }\n",
    "        )\n",
    "\n",
    "for doc in result[\"ranker\"][\"documents\"]:\n",
    "    print(\"Product ID:\", doc.meta[\"prod_id\"])\n",
    "    print(\"Product Image:\", doc.meta[\"prod_image\"])\n",
    "    print(\"Score:\", doc.score)\n",
    "    print(\"-\"*100)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "R4BF-6skZR7k",
   "metadata": {
    "id": "R4BF-6skZR7k"
   },
   "source": [
    "![image.png](./images/reranker-retrieved-results.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2L7lGgvpJSKe",
   "metadata": {
    "id": "2L7lGgvpJSKe"
   },
   "source": [
    "*Compared to BM25, Jina Reranker returns a much more relevant collection of answers. In our e-commerce setting, this translates directly to a better user experience and increased likelihood of purchases.*"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "052c5babc9894ed598d1e6c685df6480": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0bd040d1e5774fa9819bfda81c162a45": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1b58f96e6bf449948a8a8ba747f3655f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_38938aa4c9bc4965b00cf55231c3fe44",
       "IPY_MODEL_3e1b3bf8208044edbb545d482c4179c7",
       "IPY_MODEL_2fd9aa10863f4ce9aa7851aa28451c69"
      ],
      "layout": "IPY_MODEL_dabc3892bc50408f926229dee474fbcb"
     }
    },
    "1eb03a5ec71c422f9e84b8f87e8cf87e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2a9652e7558a48d48eb2821ea49de1c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f219657745f341d7b03b812d990f2b10",
       "IPY_MODEL_9fb25fb75bd54aab971304bbc6210a8f",
       "IPY_MODEL_4b19c42c412b46e2bd819f305c6f076d"
      ],
      "layout": "IPY_MODEL_c511e21371fa4d388fcfd9d9e4618621"
     }
    },
    "2fd9aa10863f4ce9aa7851aa28451c69": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f7170b0156324004bcc8eeff0ac8697a",
      "placeholder": "​",
      "style": "IPY_MODEL_fc9abd42e5d24b538fb60e6c3d7233fb",
      "value": " 27500/27500 [00:02<00:00, 7693.45 docs/s]"
     }
    },
    "38938aa4c9bc4965b00cf55231c3fe44": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_feaf0bcb4ce54950bbe027e97d29e7fe",
      "placeholder": "​",
      "style": "IPY_MODEL_0bd040d1e5774fa9819bfda81c162a45",
      "value": "Ranking by BM25...: 100%"
     }
    },
    "3e1b3bf8208044edbb545d482c4179c7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5fe3ba5614264d43888c61b1e4a1e93c",
      "max": 27500,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_052c5babc9894ed598d1e6c685df6480",
      "value": 27500
     }
    },
    "4b19c42c412b46e2bd819f305c6f076d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d11e0276f5ec4169b534013d430a1446",
      "placeholder": "​",
      "style": "IPY_MODEL_bf8db219aba845f2abd7a80b9dc9c898",
      "value": " 27500/27500 [00:03<00:00, 8720.99 docs/s]"
     }
    },
    "4fe2fef4428e4105bc2a615ab4ae878d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "55756a04a1d94c63a69439b8195314a9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5fe3ba5614264d43888c61b1e4a1e93c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9fb25fb75bd54aab971304bbc6210a8f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1eb03a5ec71c422f9e84b8f87e8cf87e",
      "max": 27500,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d01408fff8a945bf9197238b58c5dd40",
      "value": 27500
     }
    },
    "bf8db219aba845f2abd7a80b9dc9c898": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c511e21371fa4d388fcfd9d9e4618621": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d01408fff8a945bf9197238b58c5dd40": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d11e0276f5ec4169b534013d430a1446": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dabc3892bc50408f926229dee474fbcb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f219657745f341d7b03b812d990f2b10": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_55756a04a1d94c63a69439b8195314a9",
      "placeholder": "​",
      "style": "IPY_MODEL_4fe2fef4428e4105bc2a615ab4ae878d",
      "value": "Ranking by BM25...: 100%"
     }
    },
    "f7170b0156324004bcc8eeff0ac8697a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fc9abd42e5d24b538fb60e6c3d7233fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "feaf0bcb4ce54950bbe027e97d29e7fe": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}