{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e69ba12f-305c-4e43-aa02-54a09093c321",
   "metadata": {},
   "source": [
    "<h1>Text Extensions for Pandas</h1>\n",
    "<h2>Interactive Dataframe Widget</h2>\n",
    "The interactive dataframe widget is an application within the IBM CODAIT team's open source Python library: Text Extension for Pandas. The widget aims to provide data scientists with a meaningful, visual way to interpret NLP (Natural Language Processing) data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d23a275a-09ac-4de0-9e07-1a71adb78365",
   "metadata": {},
   "source": [
    "This demo will walk you though an example session of using the widget and related visualizers provided in the ```jupyter``` sub-module of Text Extensions for Pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8a02abec-ae6b-4ad8-903b-f182418726e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import regex\n",
    "import sys\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "# And of course we need the text_extensions_for_pandas library itself.\n",
    "try:\n",
    "    import text_extensions_for_pandas as tp\n",
    "except ModuleNotFoundError as e:\n",
    "    # If we're running from within the project source tree and the parent Python\n",
    "    # environment doesn't have the text_extensions_for_pandas package, use the\n",
    "    # version in the local source tree.\n",
    "    if not os.getcwd().endswith(\"notebooks\"):\n",
    "        raise e\n",
    "    if \"..\" not in sys.path:\n",
    "        sys.path.insert(0, \"..\")\n",
    "    import text_extensions_for_pandas as tp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f101deb-5c29-4ee2-be0c-da7061d0b5c9",
   "metadata": {},
   "source": [
    "This demo will make use of the CoNLL-2003 dataset, a dataset concerning named entity recognition (Named Entity Extraction). We will be looking at a token classification problem - analyzing the building blocks of natural language present in this dataset that we can process and feed into a machine learning algorithm. The dataset contains categorical entity classifications of ```locations (LOC)```, ```persons (PER)```, ```organizations (ORG)``` and ```miscellaneous (MISC)```.\n",
    "\n",
    "Our goal is to load up some data from this dataset and do some basic processing and analysis, and make corrections if necessary.\n",
    "\n",
    "We will use Text Extensions for Pandas to download and parse the CoNLL dataset into dataframes to work with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ac6e58cc-57fb-4d71-ba64-364fd2255d95",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'train': 'outputs/eng.train',\n",
       " 'dev': 'outputs/eng.testa',\n",
       " 'test': 'outputs/eng.testb'}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Download and cache the data set.\n",
    "# NOTE: This data set is licensed for research use only. Be sure to adhere\n",
    "#  to the terms of the license when using this data set!\n",
    "data_set_info = tp.io.conll.maybe_download_conll_data(\"outputs\")\n",
    "data_set_info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2847afc9-6e3e-48a9-acc0-326cdf45877d",
   "metadata": {},
   "outputs": [],
   "source": [
    "gold_standard = tp.io.conll.conll_2003_to_dataframes(\n",
    "    data_set_info[\"test\"], [\"pos\", \"phrase\", \"ent\"], [False, True, True])\n",
    "gold_standard = [\n",
    "    df.drop(columns=[\"pos\", \"phrase_iob\", \"phrase_type\"])\n",
    "    for df in gold_standard\n",
    "]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3dd3e7b-6535-4476-8e8a-997b2ba5e0d0",
   "metadata": {},
   "source": [
    "Once we have our dataset downloaded and parsed, we can prepare our dataframe for visualization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3651acbd-18e8-45e8-bd5c-9427659e2fd3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>span</th>\n",
       "      <th>ent_iob</th>\n",
       "      <th>ent_type</th>\n",
       "      <th>sentence</th>\n",
       "      <th>line_num</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[0, 10): '-DOCSTART-'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[0, 10): '-DOCSTART-'</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[11, 17): 'SOCCER'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[17, 18): '-'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[19, 24): 'JAPAN'</td>\n",
       "      <td>B</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[25, 28): 'GET'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>415</th>\n",
       "      <td>[2178, 2182): 'each'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[2138, 2197): 'All four teams are level with o...</td>\n",
       "      <td>437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>416</th>\n",
       "      <td>[2183, 2187): 'from'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[2138, 2197): 'All four teams are level with o...</td>\n",
       "      <td>438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>417</th>\n",
       "      <td>[2188, 2191): 'one'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[2138, 2197): 'All four teams are level with o...</td>\n",
       "      <td>439</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>418</th>\n",
       "      <td>[2192, 2196): 'game'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[2138, 2197): 'All four teams are level with o...</td>\n",
       "      <td>440</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>419</th>\n",
       "      <td>[2196, 2197): '.'</td>\n",
       "      <td>O</td>\n",
       "      <td>None</td>\n",
       "      <td>[2138, 2197): 'All four teams are level with o...</td>\n",
       "      <td>441</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>420 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      span ent_iob ent_type  \\\n",
       "0    [0, 10): '-DOCSTART-'       O     None   \n",
       "1       [11, 17): 'SOCCER'       O     None   \n",
       "2            [17, 18): '-'       O     None   \n",
       "3        [19, 24): 'JAPAN'       B      LOC   \n",
       "4          [25, 28): 'GET'       O     None   \n",
       "..                     ...     ...      ...   \n",
       "415   [2178, 2182): 'each'       O     None   \n",
       "416   [2183, 2187): 'from'       O     None   \n",
       "417    [2188, 2191): 'one'       O     None   \n",
       "418   [2192, 2196): 'game'       O     None   \n",
       "419      [2196, 2197): '.'       O     None   \n",
       "\n",
       "                                              sentence  line_num  \n",
       "0                                [0, 10): '-DOCSTART-'         0  \n",
       "1    [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...         2  \n",
       "2    [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...         3  \n",
       "3    [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...         4  \n",
       "4    [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...         5  \n",
       "..                                                 ...       ...  \n",
       "415  [2138, 2197): 'All four teams are level with o...       437  \n",
       "416  [2138, 2197): 'All four teams are level with o...       438  \n",
       "417  [2138, 2197): 'All four teams are level with o...       439  \n",
       "418  [2138, 2197): 'All four teams are level with o...       440  \n",
       "419  [2138, 2197): 'All four teams are level with o...       441  \n",
       "\n",
       "[420 rows x 5 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens = gold_standard[0]\n",
    "tokens"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d1c602a3-186a-4059-847b-753e73df685e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>span</th>\n",
       "      <th>ent_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[19, 24): 'JAPAN'</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[40, 45): 'CHINA'</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[66, 77): 'Nadim Ladki'</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[78, 84): 'AL-AIN'</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[86, 106): 'United Arab Emirates'</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                span ent_type\n",
       "0                  [19, 24): 'JAPAN'      LOC\n",
       "1                  [40, 45): 'CHINA'      PER\n",
       "2            [66, 77): 'Nadim Ladki'      PER\n",
       "3                 [78, 84): 'AL-AIN'      LOC\n",
       "4  [86, 106): 'United Arab Emirates'      LOC"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "entity_mentions = tp.io.conll.iob_to_spans(tokens)\n",
    "entity_mentions.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "2e539ae1-b4f6-4cc1-bb74-adb206e32544",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>span</th>\n",
       "      <th>ent_type</th>\n",
       "      <th>sentence</th>\n",
       "      <th>sentence_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[19, 24): 'JAPAN'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[40, 45): 'CHINA'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[66, 77): 'Nadim Ladki'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[66, 77): 'Nadim Ladki'</td>\n",
       "      <td>66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[78, 84): 'AL-AIN'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[78, 117): 'AL-AIN, United Arab Emirates 1996-...</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[86, 106): 'United Arab Emirates'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[78, 117): 'AL-AIN, United Arab Emirates 1996-...</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                span ent_type  \\\n",
       "0                  [19, 24): 'JAPAN'      LOC   \n",
       "1                  [40, 45): 'CHINA'      PER   \n",
       "2            [66, 77): 'Nadim Ladki'      PER   \n",
       "3                 [78, 84): 'AL-AIN'      LOC   \n",
       "4  [86, 106): 'United Arab Emirates'      LOC   \n",
       "\n",
       "                                            sentence  sentence_id  \n",
       "0  [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...           11  \n",
       "1  [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...           11  \n",
       "2                            [66, 77): 'Nadim Ladki'           66  \n",
       "3  [78, 117): 'AL-AIN, United Arab Emirates 1996-...           78  \n",
       "4  [78, 117): 'AL-AIN, United Arab Emirates 1996-...           78  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sentences = tokens[\"sentence\"].unique()\n",
    "entity_sentence_pairs = tp.spanner.contain_join(pd.Series(sentences), entity_mentions[\"span\"], \"sentence\", \"span\")\n",
    "entity_mentions = entity_mentions.merge(entity_sentence_pairs)\n",
    "entity_mentions[\"sentence_id\"] = entity_mentions[\"sentence\"].array.begin\n",
    "entity_mentions.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cd6d543-df21-4774-8140-5d45d6a1b7ca",
   "metadata": {},
   "source": [
    "We can take a closer look at what the ```span``` column might look like in context by viewing the column alone as the SpanArray datatype."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e777948a-ffca-4a4c-9a04-31ac90a88698",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<style class=\"span-array-css\">\n",
       "            .span-array {\n",
       "    --thead-background-color: var(--jp-layout-color1, inherit);\n",
       "    --thead-text-color: var(--jp-ui-font-color1, inherit);\n",
       "    --tbody-background-color-1: var(--jp-layout-color1, inherit);\n",
       "    --tbody-background-color-2: var(--jp-layout-color2, inherit);\n",
       "    --tbody-background-color-hover: var(--jp-rendermime-table-row-hover-background, var(--jp-layout-color3, inherit));\n",
       "    --tbody-background-color-disabled: var(--jp-layout-color4, #ccccd1);\n",
       "    --tbody-text-color: var(--jp-ui-font-color0, inherit);\n",
       "    --tbody-text-color-disabled: var(--jp-ui-inverse-font-color0, #b3b3b9);\n",
       "    --table-font-family: var(--jp-content-font-family, var(--fallback-font-family, inherit));\n",
       "\n",
       "    --table-control-background: rgba(0, 0, 0, 0.2);\n",
       "    --table-control-color: var(--jp-ui-font-color0);\n",
       "    --table-control-border: 1px solid rgba(0, 0, 0, 0.8);\n",
       "    --table-control-border-radius: 0.5em;\n",
       "\n",
       "    --root-highlight: #a0c4ff;\n",
       "    --nested-highlight: #ffadad;\n",
       "    --hover-highlight: #ffd6a5;\n",
       "\n",
       "    --inverted-background-color: #0B525B;\n",
       "    --inverted-text-color: rgb(243, 243, 243);\n",
       "    --paragraph-border-color: var(--jp-layout-color2, inherit);\n",
       "\n",
       "    --fallback-font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif\n",
       "}\n",
       "\n",
       "/* Table of span offsets */\n",
       ".span-array>.document>table {\n",
       "    table-layout: auto;\n",
       "    overflow: hidden;\n",
       "    width: 100%;\n",
       "    border-collapse: collapse;\n",
       "    font-family: var(--table-font-family);\n",
       "}\n",
       "\n",
       ".span-array>.document>table thead {\n",
       "    font-variant-caps: all-petite-caps;\n",
       "}\n",
       "\n",
       ".span-array>.document>table th {\n",
       "    padding: 1em;\n",
       "}\n",
       "\n",
       ".span-array>.document>table tr>td:last-child, .span-array>.document>table tr>th:last-child {\n",
       "    text-align: right;\n",
       "    width: 100%;\n",
       "}\n",
       "\n",
       ".span-array>.document>table tr>td:not(tr>td:last-child), .span-array>.document>table tr>th:not(tr>th:last-child) {\n",
       "    text-align: left;\n",
       "}\n",
       "\n",
       ".span-array>.document>table tr.disabled:nth-child(n), .span-array>.document>table tr.disabled.hover:nth-child(n) {\n",
       "    background-color: var(--tbody-background-color-disabled);\n",
       "}\n",
       "\n",
       ".span-array>.document>table tr.disabled:nth-child(n)>td, .span-array>.document>table tr.disabled.hover:nth-child(n)>td {\n",
       "    color: var(--tbody-text-color-disabled);\n",
       "}\n",
       "\n",
       ".span-array>.document>table tr.hover:not(.disabled) {\n",
       "    background: var(--jp-rendermime-table-row-hover-background);\n",
       "}\n",
       "\n",
       "/* Table control buttons */\n",
       "\n",
       ".span-array>.document>table td.sa-table-controls-container {\n",
       "    vertical-align: center;\n",
       "}\n",
       "\n",
       ".span-array>.document>table div.sa-table-controls {\n",
       "    display: flex;\n",
       "    flex-direction: row;\n",
       "}\n",
       "\n",
       ".span-array>.document>table div.sa-table-controls button {\n",
       "    background-color: var(--table-control-background);\n",
       "    color: var(--table-control-color);\n",
       "    border: var(--table-control-border);\n",
       "    border-right: none;\n",
       "    border-radius: 0;\n",
       "    cursor: pointer;\n",
       "}\n",
       "\n",
       ".span-array>.document>table div.sa-table-controls button:first-child {\n",
       "    border-radius: var(--table-control-border-radius) 0 0 var(--table-control-border-radius);\n",
       "}\n",
       "\n",
       ".span-array>.document>table div.sa-table-controls button:last-child {\n",
       "    border-radius: 0 var(--table-control-border-radius) var(--table-control-border-radius) 0;\n",
       "    border-right: var(--table-control-border);\n",
       "}\n",
       "\n",
       ".span-array>.document>table div.sa-table-controls button[data-control=\"visibility\"]:hover {\n",
       "    background-color: var(--root-highlight);\n",
       "    color: black;\n",
       "}\n",
       "\n",
       ".span-array>.document>table tr:not(tr.disabled) div.sa-table-controls button[data-control=\"highlight\"]:hover {\n",
       "    background-color: var(--hover-highlight);\n",
       "    color: black;\n",
       "}\n",
       "\n",
       ".span-array>.document>table tr.highlighted:not(tr.disabled) div.sa-table-controls button[data-control=\"highlight\"] {\n",
       "    background-color: var(--hover-highlight);\n",
       "    color: black;\n",
       "}\n",
       "\n",
       "/* Styling for spans within document context */\n",
       ".span-array>.document>p {\n",
       "    border:1px solid var(--paragraph-border-color);\n",
       "    border-radius: 0.2em;\n",
       "    padding: 1em;\n",
       "    line-height: calc(var(--jp-content-line-height, 1.6) * 1.6);\n",
       "    box-sizing: border-box;\n",
       "    font-family: var(--jp-content-font-family, var(--fallback-font-family, inherit));\n",
       "}\n",
       "\n",
       "body[data-jp-theme-light=\"false\"].span-array>.document>p {\n",
       "    border: 1px solid black;\n",
       "    background-color: var(--inverted-background-color);\n",
       "    color: var(--inverted-text-color);\n",
       "}\n",
       "\n",
       ".span-array>.document>p mark {\n",
       "    padding: 0.4em 0.4em;\n",
       "    border-radius: 0.35em;\n",
       "    cursor: pointer;\n",
       "}\n",
       "\n",
       ".span-array>.document>p .mark {\n",
       "    color: var(black);\n",
       "}\n",
       "\n",
       ".span-array>.document>p mark {\n",
       "    background-color: var(--root-highlight);\n",
       "}\n",
       "\n",
       ".span-array>.document>p mark>mark {\n",
       "    background-color: var(--nested-highlight);\n",
       "    padding: 0.2em 0.4em;\n",
       "}\n",
       "\n",
       ".span-array>.document>p mark.complex-set {\n",
       "    background: linear-gradient(to right, var(--root-highlight), var(--nested-highlight))\n",
       "}\n",
       "\n",
       ".span-array>.document>p mark>span.mark-tag {\n",
       "    font-weight: bolder;\n",
       "    font-size: 0.8em;\n",
       "    font-variant: small-caps;\n",
       "    font-variant-caps: all-small-caps;\n",
       "    margin-left: 8px;\n",
       "    text-transform: uppercase;\n",
       "}\n",
       "\n",
       ".span-array>.document>p mark.hover, .span-array.span-array>.document>p mark>mark.hover, .span-array>.document>p mark.complex-set.hover, .span-array>.document>p mark.highlighted, .span-array>.document>p mark.complex-set.highlighted, .span-array.span-array>.document>p mark>mark.highlighted {\n",
       "    background: none;\n",
       "    background-color: var(--hover-highlight);\n",
       "}\n",
       "\n",
       "</style>\n",
       "<script>\n",
       "{\n",
       "            // Increment the version to invalidate the cached script\n",
       "const VERSION = 0.79\n",
       "const global_stylesheet = document.head.querySelector(\"style.span-array-css\")\n",
       "const local_stylesheet = document.currentScript.parentElement.querySelector(\"style.span-array-css\")\n",
       "\n",
       "if(window.SpanArray == undefined || window.SpanArray.VERSION == undefined || window.SpanArray.VERSION < VERSION) {\n",
       "\n",
       "    // Replace global SpanArray CSS with latest copy\n",
       "    if(local_stylesheet != undefined) {\n",
       "        if(global_stylesheet != undefined) {\n",
       "            document.head.removeChild(global_stylesheet)\n",
       "        }\n",
       "        document.head.appendChild(local_stylesheet)\n",
       "    }\n",
       "\n",
       "    // Sets up the SpanArray global namespace\n",
       "    window.SpanArray = {}\n",
       "    window.SpanArray.VERSION = VERSION\n",
       "\n",
       "    window.SpanArray.TYPE_OVERLAP = 0;\n",
       "    window.SpanArray.TYPE_NESTED = 1;\n",
       "    window.SpanArray.TYPE_COMPLEX = 2;\n",
       "    window.SpanArray.TYPE_SOLO = 3;\n",
       "\n",
       "    const TYPE_OVERLAP = window.SpanArray.TYPE_OVERLAP;\n",
       "    const TYPE_NESTED = window.SpanArray.TYPE_NESTED;\n",
       "    const TYPE_COMPLEX = window.SpanArray.TYPE_COMPLEX;\n",
       "    const TYPE_SOLO = window.SpanArray.TYPE_SOLO;\n",
       "\n",
       "    function sanitize(input) {\n",
       "        let out = input.slice();\n",
       "        out = out.replace(/&/g, \"&amp;\")\n",
       "        out = out.replace(/</g, \"&lt;\")\n",
       "        out = out.replace(/>/g, \"&gt;\")\n",
       "        out = out.replace(/\\$/g, \"<span>&#36;</span>\")\n",
       "        out = out.replace(/\"/g, \"&quot;\")\n",
       "        out = out.replace(/(\\r|\\n)/g, \"<br>\")\n",
       "        return out;\n",
       "    }\n",
       "\n",
       "    /** Comparison function used to sort SpanArrays by position and length\n",
       "     *  Will sort by primarily by earliest beginning point. On a tie, will prioritize latest end point (first and largest)\n",
       "     *  Used by mark relationship algorithm.\n",
       "     */\n",
       "    function compareSpanArrays(a, b) {\n",
       "        const start_diff = a.begin - b.begin\n",
       "        if(start_diff == 0) {\n",
       "            return b.end - a.end\n",
       "        }\n",
       "        return start_diff;\n",
       "    }\n",
       "\n",
       "    /** Models an instance of a SpanArray, with document-separated spans and text\n",
       "     * NOTE: Using docs instead of documents to avoid unintentionally manipulating the global 'document' object.\n",
       "    */\n",
       "    class SpanArray {\n",
       "        constructor(docs, show_offsets, script_context) {\n",
       "            this.docs = docs\n",
       "            this.show_offsets = show_offsets\n",
       "            this.script_context = script_context\n",
       "\n",
       "            // For each doc, generate a lookup map for quick ID access\n",
       "            this.docs = this.docs.map(doc => {\n",
       "                doc.span_objects = Span.arrayFromSpanArray(doc.doc_spans)\n",
       "                doc.lookup_table = {}\n",
       "                doc.span_objects.forEach(span => {\n",
       "                    doc.lookup_table[span.id] = span\n",
       "                })\n",
       "                return doc\n",
       "            })\n",
       "        }\n",
       "\n",
       "        render() {\n",
       "            let span_array_frag = document.createDocumentFragment()\n",
       "            // For each document, create a document fragment and append to a document container\n",
       "            for(let doc_index = 0; doc_index < this.docs.length; doc_index++)\n",
       "            {\n",
       "                let doc = this.docs[doc_index]\n",
       "                let doc_container = document.createElement(\"div\")\n",
       "                // Using the data-doc-id attribute allows a selector to access a document's render by its index\n",
       "                doc_container.setAttribute(\"data-doc-id\", doc_index)\n",
       "                doc_container.classList.add(\"document\")\n",
       "                const document_fragment = getDocumentFragment(doc, this.show_offsets)\n",
       "                if(this.show_offsets) {\n",
       "                    attachDocumentEvents(document_fragment, doc, this)\n",
       "                }\n",
       "                doc_container.appendChild(document_fragment)\n",
       "                span_array_frag.appendChild(doc_container)\n",
       "            }\n",
       "            let container = this.script_context.parentElement.querySelector(\".span-array\")\n",
       "            if(container != undefined) {\n",
       "                container.innerHTML = \"\"\n",
       "                container.appendChild(span_array_frag)\n",
       "            } else {\n",
       "                console.error(\"No container found for SpanArray renderer\")\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "\n",
       "    window.SpanArray.SpanArray = SpanArray\n",
       "\n",
       "\n",
       "    /** Models an instance of a Span and its relationship to other spans in the document */\n",
       "    class Span {\n",
       "\n",
       "        // Creates an ordered list of entries from a list of spans with struct [begin, end]\n",
       "        static arrayFromSpanArray(spanArray) {\n",
       "            let entries = []\n",
       "            let span;\n",
       "            for(let i = 0; i < spanArray.length; i++)\n",
       "            {\n",
       "                span = spanArray[i];\n",
       "                entries.push(new Span(i, span[0], span[1]))\n",
       "            }\n",
       "\n",
       "            entries = entries.sort(compareSpanArrays)\n",
       "\n",
       "            let set;\n",
       "            for(let i = 0; i < entries.length; i++) {\n",
       "                for(let j = i+1; j < entries.length && entries[j].begin < entries[i].end; j++) {\n",
       "                    if(entries[j].end <= entries[i].end) {\n",
       "                        set = {type: TYPE_NESTED, entry: entries[j]}\n",
       "                    } else {\n",
       "                        set = {type: TYPE_OVERLAP, entry: entries[j]}\n",
       "                    }\n",
       "                    entries[i].sets.push(set)\n",
       "                }\n",
       "            }\n",
       "\n",
       "            return entries\n",
       "        }\n",
       "\n",
       "        constructor(id, begin, end) {\n",
       "            this.id = id\n",
       "            this.begin = begin\n",
       "            this.end = end\n",
       "            this.sets = []\n",
       "            this.visible = true\n",
       "            this.highlighted = false\n",
       "        }\n",
       "\n",
       "        // Returns only visible sets\n",
       "        get valid_sets() {\n",
       "            let valid_sets = []\n",
       "\n",
       "            this.sets.forEach(set => {\n",
       "                if(set.entry.visible) valid_sets.push(set)\n",
       "            })\n",
       "\n",
       "            return valid_sets\n",
       "        }\n",
       "\n",
       "        // Returns true if mark should render as a compound set of spans\n",
       "        isComplex() {\n",
       "            for(let i = 0; i < this.valid_sets.length; i++) {\n",
       "                let otherMember = this.valid_sets[i].entry;\n",
       "                if(this.valid_sets[i].type == TYPE_OVERLAP && otherMember.visible) {\n",
       "                    return true;\n",
       "                } else {\n",
       "                    if(otherMember.valid_sets.length > 0 && otherMember.visible) {\n",
       "                        return true;\n",
       "                    }\n",
       "                }\n",
       "            }\n",
       "            return false;\n",
       "        }\n",
       "\n",
       "        // Gets the combined span of all connected elements\n",
       "        getSetSpan() {\n",
       "            let begin = this.begin\n",
       "            let end = this.end\n",
       "            let highest_id = this.id\n",
       "\n",
       "            this.valid_sets.forEach(set => {\n",
       "                let other = set.entry.getSetSpan()\n",
       "                if(other.begin < begin) begin = other.begin\n",
       "                if(other.end > end) end = other.end\n",
       "                if(other.highest_id > highest_id) highest_id = other.highest_id\n",
       "            })\n",
       "\n",
       "            return {begin: begin, end: end, highest_id: highest_id}\n",
       "        }\n",
       "    }\n",
       "\n",
       "    window.SpanArray.Span = Span\n",
       "\n",
       "    // Get the DocumentFragment for a single document \n",
       "    function getDocumentFragment(doc, show_offsets) {\n",
       "\n",
       "        const doc_text = doc.doc_text;\n",
       "        const entries = doc.span_objects;\n",
       "\n",
       "        let frag = document.createDocumentFragment()\n",
       "\n",
       "        // Render Table\n",
       "        if(show_offsets) {\n",
       "            let table = document.createElement(\"table\")\n",
       "            table.innerHTML = `\n",
       "            <thead>\n",
       "            <tr>\n",
       "                <th></th>\n",
       "                <th></th>\n",
       "                <th>begin</th>\n",
       "                <th>end</th>\n",
       "                ${(doc['doc_token_spans'] != undefined) ? '<th>begin token</th> <th>end token</th>' : ''}\n",
       "                <th>context</th>\n",
       "            </tr>\n",
       "            </thead>`\n",
       "            let tbody = document.createElement(\"tbody\")\n",
       "            entries.forEach(entry => {\n",
       "                let row = document.createElement(\"tr\")\n",
       "                row.setAttribute(\"data-id\", entry.id.toString())\n",
       "                if(!entry.visible)\n",
       "                {\n",
       "                    row.classList.add(\"disabled\")\n",
       "                }\n",
       "                if(entry.highlighted)\n",
       "                {\n",
       "                    row.classList.add(\"highlighted\")\n",
       "                }\n",
       "\n",
       "                // Adds the span entry to the table. doc_text is sanitized by replacing the reserved\n",
       "                // symbols by their entity name representations\n",
       "                row.innerHTML += `\n",
       "                <td>\n",
       "                    <div class='sa-table-controls'>\n",
       "                    <button data-control='visibility' style='width:1em'><svg style='display:block;margin:0.2em auto;' xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 576 512\"><path d=\"M572.52 241.4C518.29 135.59 410.93 64 288 64S57.68 135.64 3.48 241.41a32.35 32.35 0 0 0 0 29.19C57.71 376.41 165.07 448 288 448s230.32-71.64 284.52-177.41a32.35 32.35 0 0 0 0-29.19zM288 400a144 144 0 1 1 144-144 143.93 143.93 0 0 1-144 144zm0-240a95.31 95.31 0 0 0-25.31 3.79 47.85 47.85 0 0 1-66.9 66.9A95.78 95.78 0 1 0 288 160z\"/></svg></button>\n",
       "                    <button data-control='highlight' style='width:1em'><svg style='display:block;margin:0.2em auto;' xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\"><path d=\"M256 160c-52.9 0-96 43.1-96 96s43.1 96 96 96 96-43.1 96-96-43.1-96-96-96zm246.4 80.5l-94.7-47.3 33.5-100.4c4.5-13.6-8.4-26.5-21.9-21.9l-100.4 33.5-47.4-94.8c-6.4-12.8-24.6-12.8-31 0l-47.3 94.7L92.7 70.8c-13.6-4.5-26.5 8.4-21.9 21.9l33.5 100.4-94.7 47.4c-12.8 6.4-12.8 24.6 0 31l94.7 47.3-33.5 100.5c-4.5 13.6 8.4 26.5 21.9 21.9l100.4-33.5 47.3 94.7c6.4 12.8 24.6 12.8 31 0l47.3-94.7 100.4 33.5c13.6 4.5 26.5-8.4 21.9-21.9l-33.5-100.4 94.7-47.3c13-6.5 13-24.7.2-31.1zm-155.9 106c-49.9 49.9-131.1 49.9-181 0-49.9-49.9-49.9-131.1 0-181 49.9-49.9 131.1-49.9 181 0 49.9 49.9 49.9 131.1 0 181z\"/></svg></button>\n",
       "                    </div>\n",
       "                </td>\n",
       "                <td><b>${entry.id.toString()}</b></td>\n",
       "                <td>${entry.begin}</td>\n",
       "                <td>${entry.end}</td>\n",
       "                ${(doc.doc_token_spans != undefined) ? `<td>${doc.doc_token_spans[entry.id][0]}</td><td>${doc.doc_token_spans[entry.id][1]}</td>` : ''}\n",
       "                <td>${sanitize(doc_text.substring(entry.begin, entry.end))}</td>`\n",
       "\n",
       "                tbody.appendChild(row)\n",
       "            })\n",
       "            table.appendChild(tbody)\n",
       "            frag.appendChild(table)\n",
       "        }\n",
       "\n",
       "        // Render Text\n",
       "        let highlight_regions = []\n",
       "        for(let i = 0; i < entries.length; i++)\n",
       "        {\n",
       "            if(!entries[i].visible) continue\n",
       "            if(entries[i].valid_sets.length > 0)\n",
       "            {\n",
       "                let span = entries[i].getSetSpan();\n",
       "                let ids = [entries[i].id, ...entries[i].valid_sets.map(set => { return set.entry.id })]\n",
       "                if(entries[i].isComplex()) {\n",
       "                    highlight_regions.push({begin: span.begin, end: span.end, type: TYPE_COMPLEX, ids: ids})\n",
       "                } else {\n",
       "                    highlight_regions.push({begin: span.begin, end: span.end, type: TYPE_NESTED, ids: ids})\n",
       "                }\n",
       "                i = span.highest_id\n",
       "            } else {\n",
       "                highlight_regions.push({begin: entries[i].begin, end: entries[i].end, type: TYPE_SOLO, ids: [entries[i].id]})\n",
       "            }\n",
       "        }\n",
       "\n",
       "        let paragraph = document.createElement(\"p\")\n",
       "        if(highlight_regions.length == 0) {\n",
       "            paragraph.innerHTML = sanitize(doc_text)\n",
       "        } else {\n",
       "            let begin = 0\n",
       "            highlight_regions.forEach(region => {\n",
       "                paragraph.innerHTML += sanitize(doc_text.substring(begin, region.begin))\n",
       "\n",
       "                let mark = document.createElement(\"mark\")\n",
       "                // The data-ids tag is a list of comma-separated reference IDs for matching Spans \n",
       "                mark.setAttribute(\"data-ids\", \"\");\n",
       "                if (region.type != TYPE_NESTED) {\n",
       "                    region.ids.forEach(id => {\n",
       "                        mark.setAttribute(\"data-ids\", mark.getAttribute(\"data-ids\") + `#${id},`)\n",
       "                        if(doc.lookup_table[id].highlighted) mark.classList.add(\"highlighted\")\n",
       "                    })\n",
       "                    mark.innerHTML = sanitize(doc_text.substring(region.begin, region.end))\n",
       "                } else {\n",
       "                    mark.setAttribute(\"data-ids\", `#${region.ids[0]},`)\n",
       "                    if(doc.lookup_table[region.ids[0]].highlighted) mark.classList.add(\"highlighted\")\n",
       "                    let nested_begin = region.begin\n",
       "                    region.ids.slice(1).forEach(nested_id => {\n",
       "                        let nested_region = doc.lookup_table[nested_id]\n",
       "                        mark.innerHTML += sanitize(doc_text.substring(nested_begin, nested_region.begin))\n",
       "                        let nested_mark = document.createElement(\"mark\")\n",
       "                        nested_mark.setAttribute(\"data-ids\", `#${nested_id},`)\n",
       "                        if(nested_region.highlighted) nested_mark.classList.add(\"highlighted\")\n",
       "                        nested_mark.innerHTML = sanitize(doc_text.substring(nested_region.begin, nested_region.end))\n",
       "                        nested_begin = nested_region.end\n",
       "                        mark.appendChild(nested_mark)\n",
       "                    })\n",
       "                    mark.innerHTML += sanitize(doc_text.substring(nested_begin, region.end))\n",
       "                }\n",
       "\n",
       "                if(region.type == TYPE_COMPLEX) {\n",
       "                    let markTag = document.createElement(\"span\")\n",
       "                    markTag.textContent = \"Set\"\n",
       "                    markTag.classList.add(\"mark-tag\")\n",
       "                    mark.classList.add(\"complex-set\")\n",
       "                    mark.appendChild(markTag)\n",
       "                }\n",
       "\n",
       "                begin = region.end\n",
       "                paragraph.appendChild(mark)\n",
       "            })\n",
       "            paragraph.innerHTML += sanitize(doc_text.substring(highlight_regions[highlight_regions.length - 1].end, doc_text.length))\n",
       "        }\n",
       "\n",
       "        frag.appendChild(paragraph)\n",
       "\n",
       "        return frag\n",
       "    }\n",
       "\n",
       "    /** Attach hover and click events to a document render via event delegation */\n",
       "    function attachDocumentEvents(fragment, doc_object, source_spanarray) {\n",
       "        const doc_table_body = fragment.querySelector(\"table>tbody\")\n",
       "        const doc_text = fragment.querySelector(\"p\")\n",
       "\n",
       "        // Hover highlight events\n",
       "\n",
       "        doc_table_body.addEventListener(\"pointerenter\", (event) => {\n",
       "            if(event.target.nodeName == \"TR\") {\n",
       "                event.target.classList.add(\"hover\")\n",
       "                const span_id = event.target.getAttribute(\"data-id\")\n",
       "                const marks = doc_text.querySelectorAll(\"mark[data-ids]\")\n",
       "                Array.from(marks)\n",
       "                    .filter(mark => {\n",
       "                        return mark.getAttribute(\"data-ids\").includes(`#${span_id},`)\n",
       "                    })\n",
       "                    .forEach(related_mark => {\n",
       "                        related_mark.classList.add(\"hover\")\n",
       "                    })\n",
       "            }\n",
       "        }, true)\n",
       "\n",
       "        doc_table_body.addEventListener(\"pointerleave\", (event) => {\n",
       "            if(event.target.nodeName == \"TR\") {\n",
       "                event.target.classList.remove(\"hover\")\n",
       "                const span_id = event.target.getAttribute(\"data-id\")\n",
       "                const marks = doc_text.querySelectorAll(\"mark[data-ids]\")\n",
       "                Array.from(marks)\n",
       "                    .filter(mark => {\n",
       "                        return mark.getAttribute(\"data-ids\").includes(`#${span_id},`)\n",
       "                    })\n",
       "                    .forEach(related_mark => {\n",
       "                        related_mark.classList.remove(\"hover\")\n",
       "                    })\n",
       "            }\n",
       "        }, true)\n",
       "\n",
       "        doc_text.addEventListener(\"pointerenter\", (event) => {\n",
       "            if(event.target.nodeName == \"MARK\") {\n",
       "                event.target.classList.add(\"hover\")\n",
       "                const ids = event.target.getAttribute(\"data-ids\").split(\",\").slice(0, -1)\n",
       "                Array.from(ids)\n",
       "                    .map(id_tag => {\n",
       "                        return id_tag.substring(1)\n",
       "                    })\n",
       "                    .forEach(id => {\n",
       "                        const entry = doc_table_body.querySelector(`tr[data-id=\"${id}\"]`)\n",
       "                        entry.classList.add(\"hover\")\n",
       "                    })\n",
       "            }\n",
       "        }, true)\n",
       "\n",
       "        doc_text.addEventListener(\"pointerleave\", (event) => {\n",
       "            if(event.target.nodeName == \"MARK\") {\n",
       "                event.target.classList.remove(\"hover\")\n",
       "                const ids = event.target.getAttribute(\"data-ids\").split(\",\").slice(0, -1)\n",
       "                Array.from(ids)\n",
       "                    .map(id_tag => {\n",
       "                        return id_tag.substring(1)\n",
       "                    })\n",
       "                    .forEach(id => {\n",
       "                        const entry = doc_table_body.querySelector(`tr[data-id=\"${id}\"]`)\n",
       "                        entry.classList.remove(\"hover\")\n",
       "                    })\n",
       "            }\n",
       "        }, true)\n",
       "\n",
       "        // Click disable/enable events\n",
       "\n",
       "        doc_table_body.addEventListener(\"click\", (event) => {\n",
       "            const closest_control_button = event.target.closest(\"button[data-control]\")\n",
       "            if(closest_control_button == undefined) return\n",
       "\n",
       "            const closest_tr = event.target.closest(\"tr\")\n",
       "            if(closest_tr == undefined) return\n",
       "\n",
       "            const matching_span = doc_object.lookup_table[closest_tr.getAttribute(\"data-id\")]\n",
       "            if(matching_span == undefined) return\n",
       "\n",
       "            switch(closest_control_button.getAttribute(\"data-control\")) {\n",
       "                case \"visibility\":\n",
       "                    {\n",
       "                        matching_span.visible = !matching_span.visible\n",
       "                        source_spanarray.render()\n",
       "                    }\n",
       "                    break;\n",
       "                case \"highlight\":\n",
       "                    {\n",
       "                        matching_span.highlighted = !matching_span.highlighted\n",
       "                        source_spanarray.render()\n",
       "                    }\n",
       "                    break;\n",
       "            }\n",
       "\n",
       "\n",
       "\n",
       "        }, true)\n",
       "\n",
       "        doc_text.addEventListener(\"click\", (event) => {\n",
       "            const closest_mark = event.target.closest(\"mark\")\n",
       "            if(closest_mark == undefined) return\n",
       "\n",
       "            // Preprocess ID string into a list of IDs\n",
       "            const ids = closest_mark.getAttribute(\"data-ids\")\n",
       "                .split(\",\")\n",
       "                .slice(0, -1)\n",
       "                .map(id => {\n",
       "                    return id.substring(1)\n",
       "                })\n",
       "\n",
       "            // If any of the connected IDs are highlighted, we set all spans in the list to not highlighted.\n",
       "            // Inversely, we want all spans highlighted if none were previously.\n",
       "\n",
       "            const highlighted_entry = ids.find(id => {\n",
       "                return doc_object.lookup_table[id].highlighted\n",
       "            })\n",
       "\n",
       "            const is_highlighted = (highlighted_entry != undefined)\n",
       "\n",
       "            ids.forEach(id => {\n",
       "                const span = doc_object.lookup_table[id]\n",
       "                if(span != undefined) span.highlighted = !is_highlighted\n",
       "            })\n",
       "\n",
       "            source_spanarray.render()\n",
       "        })\n",
       "    }\n",
       "} else {\n",
       "    // SpanArray JS is already defined and not an outdated copy\n",
       "    // Replace global SpanArray CSS with latest copy IFF global stylesheet is undefined\n",
       "\n",
       "    if(local_stylesheet != undefined) {\n",
       "        if(global_stylesheet == undefined) {\n",
       "            document.head.appendChild(local_stylesheet)\n",
       "        } else {\n",
       "            document.currentScript.parentElement.removeChild(local_stylesheet)\n",
       "        }\n",
       "    }       \n",
       "}\n",
       "}\n",
       "</script>\n",
       "<div class=\"span-array\">\n",
       "\n",
       "    <div class='document'>\n",
       "        <table style='\n",
       "            table-layout: auto;\n",
       "            overflow: hidden;\n",
       "            width: 100%;\n",
       "            border-collapse: collapse;\n",
       "            '>\n",
       "            <thead style='font-variant-caps: all-petite-caps;'>\n",
       "                <th></th>\n",
       "                <th>begin</th>\n",
       "                <th>end</th>\n",
       "                <th>begin token</th><th>end token</th>\n",
       "                <th style='text-align:right;width:100%'>context</th>\n",
       "            </tr></thead>\n",
       "            <tbody>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>0</b></td>\n",
       "            <td>11</td>\n",
       "            <td>65</td>\n",
       "\n",
       "            <td>1</td>\n",
       "            <td>13</td>\n",
       "\n",
       "            <td>SOCCER- JAPAN GET LUCKY WIN, CHINA IN SURPRISE DEFEAT.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>1</b></td>\n",
       "            <td>66</td>\n",
       "            <td>77</td>\n",
       "\n",
       "            <td>13</td>\n",
       "            <td>15</td>\n",
       "\n",
       "            <td>Nadim Ladki</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>2</b></td>\n",
       "            <td>78</td>\n",
       "            <td>117</td>\n",
       "\n",
       "            <td>15</td>\n",
       "            <td>21</td>\n",
       "\n",
       "            <td>AL-AIN, United Arab Emirates 1996-12-06</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>3</b></td>\n",
       "            <td>118</td>\n",
       "            <td>244</td>\n",
       "\n",
       "            <td>21</td>\n",
       "            <td>46</td>\n",
       "\n",
       "            <td>Japan began the defence of their Asian Cup title with a lucky 2-1 win against Syria in a Group C championship match on Friday.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>4</b></td>\n",
       "            <td>245</td>\n",
       "            <td>374</td>\n",
       "\n",
       "            <td>46</td>\n",
       "            <td>71</td>\n",
       "\n",
       "            <td>But China saw their luck desert them in the second match of the group, crashing to a surprise 2-0 defeat to newcomers Uzbekistan.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>5</b></td>\n",
       "            <td>375</td>\n",
       "            <td>617</td>\n",
       "\n",
       "            <td>71</td>\n",
       "            <td>113</td>\n",
       "\n",
       "            <td>China controlled most of the match and saw several chances missed until the 78th minute when Uzbek striker Igor Shkvyrin took advantage of a misdirected defensive header to lob the ball over the advancing Chinese keeper and into an empty net.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>6</b></td>\n",
       "            <td>618</td>\n",
       "            <td>735</td>\n",
       "\n",
       "            <td>113</td>\n",
       "            <td>136</td>\n",
       "\n",
       "            <td>Oleg Shatskiku made sure of the win in injury time, hitting an unstoppable left foot shot from just outside the area.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>7</b></td>\n",
       "            <td>736</td>\n",
       "            <td>821</td>\n",
       "\n",
       "            <td>136</td>\n",
       "            <td>153</td>\n",
       "\n",
       "            <td>The former Soviet republic was playing in an Asian Cup finals tie for the first time.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>8</b></td>\n",
       "            <td>822</td>\n",
       "            <td>917</td>\n",
       "\n",
       "            <td>153</td>\n",
       "            <td>171</td>\n",
       "\n",
       "            <td>Despite winning the Asian Games title two years ago, Uzbekistan are in the finals as outsiders.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>9</b></td>\n",
       "            <td>918</td>\n",
       "            <td>1078</td>\n",
       "\n",
       "            <td>171</td>\n",
       "            <td>199</td>\n",
       "\n",
       "            <td>Two goals from defensive errors in the last six minutes allowed Japan to come from behind and collect all three points from their opening meeting against Syria.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>10</b></td>\n",
       "            <td>1079</td>\n",
       "            <td>1291</td>\n",
       "\n",
       "            <td>199</td>\n",
       "            <td>237</td>\n",
       "\n",
       "            <td>Takuya Takagi scored the winner in the 88th minute, rising to head a Hiroshige Yanagimoto cross towards the Syrian goal which goalkeeper Salem Bitar appeared to have covered but then allowed to slip into the net.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>11</b></td>\n",
       "            <td>1292</td>\n",
       "            <td>1350</td>\n",
       "\n",
       "            <td>237</td>\n",
       "            <td>249</td>\n",
       "\n",
       "            <td>It was the second costly blunder by Syria in four minutes.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>12</b></td>\n",
       "            <td>1351</td>\n",
       "            <td>1502</td>\n",
       "\n",
       "            <td>249</td>\n",
       "            <td>280</td>\n",
       "\n",
       "            <td>Defender Hassan Abbas rose to intercept a long ball into the area in the 84th minute but only managed to divert it into the top corner of Bitar&#39;s goal.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>13</b></td>\n",
       "            <td>1503</td>\n",
       "            <td>1591</td>\n",
       "\n",
       "            <td>280</td>\n",
       "            <td>296</td>\n",
       "\n",
       "            <td>Nader Jokhadar had given Syria the lead with a well-struck header in the seventh minute.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>14</b></td>\n",
       "            <td>1592</td>\n",
       "            <td>1701</td>\n",
       "\n",
       "            <td>296</td>\n",
       "            <td>317</td>\n",
       "\n",
       "            <td>Japan then laid siege to the Syrian penalty area for most of the game but rarely breached the Syrian defence.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>15</b></td>\n",
       "            <td>1702</td>\n",
       "            <td>1748</td>\n",
       "\n",
       "            <td>317</td>\n",
       "            <td>326</td>\n",
       "\n",
       "            <td>Bitar pulled off fine saves whenever they did.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>16</b></td>\n",
       "            <td>1749</td>\n",
       "            <td>1820</td>\n",
       "\n",
       "            <td>326</td>\n",
       "            <td>343</td>\n",
       "\n",
       "            <td>Japan coach Shu Kamo said: &#39; &#39; The Syrian own goal proved lucky for us.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>17</b></td>\n",
       "            <td>1821</td>\n",
       "            <td>1925</td>\n",
       "\n",
       "            <td>343</td>\n",
       "            <td>363</td>\n",
       "\n",
       "            <td>The Syrians scored early and then played defensively and adopted long balls which made it hard for us. &#39;</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>18</b></td>\n",
       "            <td>1928</td>\n",
       "            <td>2049</td>\n",
       "\n",
       "            <td>364</td>\n",
       "            <td>390</td>\n",
       "\n",
       "            <td>Japan, co-hosts of the World Cup in 2002 and ranked 20th in the world by FIFA, are favourites to regain their title here.</td>\n",
       "        </tr>\n",
       "\n",
       "        <tr>\n",
       "            <td><b>19</b></td>\n",
       "            <td>2050</td>\n",
       "            <td>2137</td>\n",
       "\n",
       "            <td>390</td>\n",
       "            <td>407</td>\n",
       "\n",
       "            <td>Hosts UAE play Kuwait and South Korea take on Indonesia on Saturday in Group A matches.</td>\n",
       "        </tr>\n",
       "\n",
       "            </tbody>\n",
       "        </table>\n",
       "        <p style='\n",
       "            padding: 1em;\n",
       "            line-height: calc(var(--jp-content-line-height, 1.6) * 1.6);\n",
       "            '>\n",
       "\n",
       "            -DOCSTART-<br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>SOCCER- JAPAN GET LUCKY WIN, CHINA IN SURPRISE DEFEAT.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Nadim Ladki</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>AL-AIN, United Arab Emirates 1996-12-06</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Japan began the defence of their Asian Cup title with a lucky 2-1 win against Syria in a Group C championship match on Friday.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>But China saw their luck desert them in the second match of the group, crashing to a surprise 2-0 defeat to newcomers Uzbekistan.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>China controlled most of the match and saw several chances missed until the 78th minute when Uzbek striker Igor Shkvyrin took advantage of a misdirected defensive header to lob the ball over the advancing Chinese keeper and into an empty net.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Oleg Shatskiku made sure of the win in injury time, hitting an unstoppable left foot shot from just outside the area.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>The former Soviet republic was playing in an Asian Cup finals tie for the first time.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Despite winning the Asian Games title two years ago, Uzbekistan are in the finals as outsiders.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Two goals from defensive errors in the last six minutes allowed Japan to come from behind and collect all three points from their opening meeting against Syria.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Takuya Takagi scored the winner in the 88th minute, rising to head a Hiroshige Yanagimoto cross towards the Syrian goal which goalkeeper Salem Bitar appeared to have covered but then allowed to slip into the net.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>It was the second costly blunder by Syria in four minutes.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Defender Hassan Abbas rose to intercept a long ball into the area in the 84th minute but only managed to divert it into the top corner of Bitar&#39;s goal.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Nader Jokhadar had given Syria the lead with a well-struck header in the seventh minute.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Japan then laid siege to the Syrian penalty area for most of the game but rarely breached the Syrian defence.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Bitar pulled off fine saves whenever they did.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Japan coach Shu Kamo said: &#39; &#39; The Syrian own goal proved lucky for us.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>The Syrians scored early and then played defensively and adopted long balls which made it hard for us. &#39;</span>\n",
       "\n",
       "            <br>&#39;<br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Japan, co-hosts of the World Cup in 2002 and ranked 20th in the world by FIFA, are favourites to regain their title here.</span>\n",
       "\n",
       "            <br>\n",
       "\n",
       "                <span class='mark btn-primary' style='padding:0.4em;border-radius:0.35em;background-color: #a0c4ff;color:black;'>Hosts UAE play Kuwait and South Korea take on Indonesia on Saturday in Group A matches.</span>\n",
       "            <br>All four teams are level with one point each from one game.\n",
       "        </p>\n",
       "    </div>\n",
       "\n",
       "    <span style=\"font-size: 0.8em;color: #b3b3b3;\">Your notebook viewer does not support Javascript execution. The above rendering will not be interactive.</span>\n",
       "</div>\n",
       "<script>\n",
       "    {\n",
       "        const Span = window.SpanArray.Span\n",
       "        const script_context = document.currentScript\n",
       "        const documents = []\n",
       "\n",
       "    {\n",
       "\n",
       "    const doc_spans = [[11,65],[66,77],[78,117],[118,244],[245,374],[375,617],[618,735],[736,821],[822,917],[918,1078],[1079,1291],[1292,1350],[1351,1502],[1503,1591],[1592,1701],[1702,1748],[1749,1820],[1821,1925],[1928,2049],[2050,2137]]\n",
       "    const doc_text = '-DOCSTART-\\nSOCCER- JAPAN GET LUCKY WIN, CHINA IN SURPRISE DEFEAT.\\nNadim Ladki\\nAL-AIN, United Arab Emirates 1996-12-06\\nJapan began the defence of their Asian Cup title with a lucky 2-1 win against Syria in a Group C championship match on Friday.\\nBut China saw their luck desert them in the second match of the group, crashing to a surprise 2-0 defeat to newcomers Uzbekistan.\\nChina controlled most of the match and saw several chances missed until the 78th minute when Uzbek striker Igor Shkvyrin took advantage of a misdirected defensive header to lob the ball over the advancing Chinese keeper and into an empty net.\\nOleg Shatskiku made sure of the win in injury time, hitting an unstoppable left foot shot from just outside the area.\\nThe former Soviet republic was playing in an Asian Cup finals tie for the first time.\\nDespite winning the Asian Games title two years ago, Uzbekistan are in the finals as outsiders.\\nTwo goals from defensive errors in the last six minutes allowed Japan to come from behind and collect all three points from their opening meeting against Syria.\\nTakuya Takagi scored the winner in the 88th minute, rising to head a Hiroshige Yanagimoto cross towards the Syrian goal which goalkeeper Salem Bitar appeared to have covered but then allowed to slip into the net.\\nIt was the second costly blunder by Syria in four minutes.\\nDefender Hassan Abbas rose to intercept a long ball into the area in the 84th minute but only managed to divert it into the top corner of Bitar\\'s goal.\\nNader Jokhadar had given Syria the lead with a well-struck header in the seventh minute.\\nJapan then laid siege to the Syrian penalty area for most of the game but rarely breached the Syrian defence.\\nBitar pulled off fine saves whenever they did.\\nJapan coach Shu Kamo said: \\' \\' The Syrian own goal proved lucky for us.\\nThe Syrians scored early and then played defensively and adopted long balls which made it hard for us. \\'\\n\\'\\nJapan, co-hosts of the World Cup in 2002 and ranked 20th in the world by FIFA, are favourites to regain their title here.\\nHosts UAE play Kuwait and South Korea take on Indonesia on Saturday in Group A matches.\\nAll four teams are level with one point each from one game.'\n",
       "\n",
       "        const doc_token_spans = [[1,13],[13,15],[15,21],[21,46],[46,71],[71,113],[113,136],[136,153],[153,171],[171,199],[199,237],[237,249],[249,280],[280,296],[296,317],[317,326],[326,343],[343,363],[364,390],[390,407]]\n",
       "        documents.push({doc_text: doc_text, doc_spans: doc_spans, doc_token_spans: doc_token_spans})\n",
       "\n",
       "    }\n",
       "\n",
       "        const instance = new window.SpanArray.SpanArray(documents, true, script_context)\n",
       "        instance.render()\n",
       "    }\n",
       "</script>\n",
       "\n"
      ],
      "text/plain": [
       "<TokenSpanArray>\n",
       "[                             [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA IN SURPRISE DEFEAT.',\n",
       "                                                                         [66, 77): 'Nadim Ladki',\n",
       "                                            [78, 117): 'AL-AIN, United Arab Emirates 1996-12-06',\n",
       "       [118, 244): 'Japan began the defence of their Asian Cup title with a lucky 2-1 win [...]',\n",
       "      [245, 374): 'But China saw their luck desert them in the second match of the group, [...]',\n",
       "     [375, 617): 'China controlled most of the match and saw several chances missed until [...]',\n",
       "  [618, 735): 'Oleg Shatskiku made sure of the win in injury time, hitting an unstoppable [...]',\n",
       "   [736, 821): 'The former Soviet republic was playing in an Asian Cup finals tie for the [...]',\n",
       "  [822, 917): 'Despite winning the Asian Games title two years ago, Uzbekistan are in the [...]',\n",
       "   [918, 1078): 'Two goals from defensive errors in the last six minutes allowed Japan to [...]',\n",
       "      [1079, 1291): 'Takuya Takagi scored the winner in the 88th minute, rising to head a [...]',\n",
       "                      [1292, 1350): 'It was the second costly blunder by Syria in four minutes.',\n",
       "  [1351, 1502): 'Defender Hassan Abbas rose to intercept a long ball into the area in the [...]',\n",
       "  [1503, 1591): 'Nader Jokhadar had given Syria the lead with a well-struck header in the [...]',\n",
       " [1592, 1701): 'Japan then laid siege to the Syrian penalty area for most of the game but [...]',\n",
       "                                  [1702, 1748): 'Bitar pulled off fine saves whenever they did.',\n",
       "         [1749, 1820): 'Japan coach Shu Kamo said: ' ' The Syrian own goal proved lucky for us.',\n",
       "     [1821, 1925): 'The Syrians scored early and then played defensively and adopted long [...]',\n",
       "  [1928, 2049): 'Japan, co-hosts of the World Cup in 2002 and ranked 20th in the world by [...]',\n",
       "    [2050, 2137): 'Hosts UAE play Kuwait and South Korea take on Indonesia on Saturday in [...]']\n",
       "Length: 20, dtype: TokenSpanDtype"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "entity_mentions[\"sentence\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7a34d33-2956-4e15-ba6b-93b1fee06fd7",
   "metadata": {},
   "source": [
    "We don't really want to visualize every column in our dataframe as we're only interested in viewing the entity classifications. The next step is to drop any columns we don't care about."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84886242-284e-4a66-ad87-0f65191880bf",
   "metadata": {},
   "source": [
    "Now that our data is prepared for analysis, we can load it up in our widget."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "40f6c7f2-dd12-4038-9f14-e19d40e7301e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "480b5072a64742b2be61aaef7b151a53",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output(_dom_classes=('tep--dfwidget--output',))"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "widget = tp.jupyter.DataFrameWidget(entity_mentions.drop(columns=[\"sentence\"]))\n",
    "widget.display()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbe7bbe4-44aa-4a73-ac13-faf8bd741f2c",
   "metadata": {},
   "source": [
    "If we want to view this widget interactively, we can pass in the additional parameter ```interactive_columns``` with an array of column names we want to become interactive widgets.\n",
    "\n",
    "One thing you may notice in the above widgets is that the column ```ent_type``` is editable via a text box. This is fine, but there is a more appropriate way to interact with categorical data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "46b65ec8-865e-43eb-9990-2c91b4f298c8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d1d66f97fd2343948c14ce57bfe3aea1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output(_dom_classes=('tep--dfwidget--output',))"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical = pd.Categorical(entity_mentions[\"ent_type\"], categories=[\"PER\", \"LOC\", \"ORG\", \"MISC\"])\n",
    "entity_mentions[\"ent_type\"] = categorical\n",
    "tp.jupyter.DataFrameWidget(entity_mentions.drop(columns=[\"sentence\", \"sentence_id\"]), interactive_columns=[\"ent_type\"]).display()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "f8280a38-7c4d-45c6-b90d-60ceccc6b603",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>span</th>\n",
       "      <th>ent_type</th>\n",
       "      <th>sentence</th>\n",
       "      <th>sentence_id</th>\n",
       "      <th>new_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[19, 24): 'JAPAN'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...</td>\n",
       "      <td>11</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[40, 45): 'CHINA'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...</td>\n",
       "      <td>11</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[66, 77): 'Nadim Ladki'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[66, 77): 'Nadim Ladki'</td>\n",
       "      <td>66</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[78, 84): 'AL-AIN'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[78, 117): 'AL-AIN, United Arab Emirates 1996-...</td>\n",
       "      <td>78</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[86, 106): 'United Arab Emirates'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[78, 117): 'AL-AIN, United Arab Emirates 1996-...</td>\n",
       "      <td>78</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[118, 123): 'Japan'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[118, 244): 'Japan began the defence of their ...</td>\n",
       "      <td>118</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>[151, 160): 'Asian Cup'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[118, 244): 'Japan began the defence of their ...</td>\n",
       "      <td>118</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>[196, 201): 'Syria'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[118, 244): 'Japan began the defence of their ...</td>\n",
       "      <td>118</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>[249, 254): 'China'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[245, 374): 'But China saw their luck desert t...</td>\n",
       "      <td>245</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>[363, 373): 'Uzbekistan'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[245, 374): 'But China saw their luck desert t...</td>\n",
       "      <td>245</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>[375, 380): 'China'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[375, 617): 'China controlled most of the matc...</td>\n",
       "      <td>375</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>[468, 473): 'Uzbek'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[375, 617): 'China controlled most of the matc...</td>\n",
       "      <td>375</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>[482, 495): 'Igor Shkvyrin'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[375, 617): 'China controlled most of the matc...</td>\n",
       "      <td>375</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>[580, 587): 'Chinese'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[375, 617): 'China controlled most of the matc...</td>\n",
       "      <td>375</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>[618, 632): 'Oleg Shatskiku'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[618, 735): 'Oleg Shatskiku made sure of the w...</td>\n",
       "      <td>618</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>[747, 753): 'Soviet'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[736, 821): 'The former Soviet republic was pl...</td>\n",
       "      <td>736</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>[781, 790): 'Asian Cup'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[736, 821): 'The former Soviet republic was pl...</td>\n",
       "      <td>736</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>[842, 853): 'Asian Games'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[822, 917): 'Despite winning the Asian Games t...</td>\n",
       "      <td>822</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>[875, 885): 'Uzbekistan'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[822, 917): 'Despite winning the Asian Games t...</td>\n",
       "      <td>822</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>[982, 987): 'Japan'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[918, 1078): 'Two goals from defensive errors ...</td>\n",
       "      <td>918</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>[1072, 1077): 'Syria'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[918, 1078): 'Two goals from defensive errors ...</td>\n",
       "      <td>918</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>[1079, 1092): 'Takuya Takagi'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[1079, 1291): 'Takuya Takagi scored the winner...</td>\n",
       "      <td>1079</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>[1148, 1168): 'Hiroshige Yanagimoto'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[1079, 1291): 'Takuya Takagi scored the winner...</td>\n",
       "      <td>1079</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>[1187, 1193): 'Syrian'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[1079, 1291): 'Takuya Takagi scored the winner...</td>\n",
       "      <td>1079</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>[1216, 1227): 'Salem Bitar'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[1079, 1291): 'Takuya Takagi scored the winner...</td>\n",
       "      <td>1079</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>[1328, 1333): 'Syria'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[1292, 1350): 'It was the second costly blunde...</td>\n",
       "      <td>1292</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>[1360, 1372): 'Hassan Abbas'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[1351, 1502): 'Defender Hassan Abbas rose to i...</td>\n",
       "      <td>1351</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>[1489, 1494): 'Bitar'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[1351, 1502): 'Defender Hassan Abbas rose to i...</td>\n",
       "      <td>1351</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>[1503, 1517): 'Nader Jokhadar'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[1503, 1591): 'Nader Jokhadar had given Syria ...</td>\n",
       "      <td>1503</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>[1528, 1533): 'Syria'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[1503, 1591): 'Nader Jokhadar had given Syria ...</td>\n",
       "      <td>1503</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>[1592, 1597): 'Japan'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[1592, 1701): 'Japan then laid siege to the Sy...</td>\n",
       "      <td>1592</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>[1621, 1627): 'Syrian'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[1592, 1701): 'Japan then laid siege to the Sy...</td>\n",
       "      <td>1592</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>[1686, 1692): 'Syrian'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[1592, 1701): 'Japan then laid siege to the Sy...</td>\n",
       "      <td>1592</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>[1702, 1707): 'Bitar'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[1702, 1748): 'Bitar pulled off fine saves whe...</td>\n",
       "      <td>1702</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>[1749, 1754): 'Japan'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[1749, 1820): 'Japan coach Shu Kamo said: ' ' ...</td>\n",
       "      <td>1749</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>[1761, 1769): 'Shu Kamo'</td>\n",
       "      <td>PER</td>\n",
       "      <td>[1749, 1820): 'Japan coach Shu Kamo said: ' ' ...</td>\n",
       "      <td>1749</td>\n",
       "      <td>PER</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>[1784, 1790): 'Syrian'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[1749, 1820): 'Japan coach Shu Kamo said: ' ' ...</td>\n",
       "      <td>1749</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>[1825, 1832): 'Syrians'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[1821, 1925): 'The Syrians scored early and th...</td>\n",
       "      <td>1821</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>[1928, 1933): 'Japan'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[1928, 2049): 'Japan, co-hosts of the World Cu...</td>\n",
       "      <td>1928</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>[1951, 1960): 'World Cup'</td>\n",
       "      <td>MISC</td>\n",
       "      <td>[1928, 2049): 'Japan, co-hosts of the World Cu...</td>\n",
       "      <td>1928</td>\n",
       "      <td>MISC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>[2001, 2005): 'FIFA'</td>\n",
       "      <td>ORG</td>\n",
       "      <td>[1928, 2049): 'Japan, co-hosts of the World Cu...</td>\n",
       "      <td>1928</td>\n",
       "      <td>ORG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>[2056, 2059): 'UAE'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[2050, 2137): 'Hosts UAE play Kuwait and South...</td>\n",
       "      <td>2050</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>[2065, 2071): 'Kuwait'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[2050, 2137): 'Hosts UAE play Kuwait and South...</td>\n",
       "      <td>2050</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>[2076, 2087): 'South Korea'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[2050, 2137): 'Hosts UAE play Kuwait and South...</td>\n",
       "      <td>2050</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>[2096, 2105): 'Indonesia'</td>\n",
       "      <td>LOC</td>\n",
       "      <td>[2050, 2137): 'Hosts UAE play Kuwait and South...</td>\n",
       "      <td>2050</td>\n",
       "      <td>LOC</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    span ent_type  \\\n",
       "0                      [19, 24): 'JAPAN'      LOC   \n",
       "1                      [40, 45): 'CHINA'      PER   \n",
       "2                [66, 77): 'Nadim Ladki'      PER   \n",
       "3                     [78, 84): 'AL-AIN'      LOC   \n",
       "4      [86, 106): 'United Arab Emirates'      LOC   \n",
       "5                    [118, 123): 'Japan'      LOC   \n",
       "6                [151, 160): 'Asian Cup'     MISC   \n",
       "7                    [196, 201): 'Syria'      LOC   \n",
       "8                    [249, 254): 'China'      LOC   \n",
       "9               [363, 373): 'Uzbekistan'      LOC   \n",
       "10                   [375, 380): 'China'      LOC   \n",
       "11                   [468, 473): 'Uzbek'     MISC   \n",
       "12           [482, 495): 'Igor Shkvyrin'      PER   \n",
       "13                 [580, 587): 'Chinese'     MISC   \n",
       "14          [618, 632): 'Oleg Shatskiku'      PER   \n",
       "15                  [747, 753): 'Soviet'     MISC   \n",
       "16               [781, 790): 'Asian Cup'     MISC   \n",
       "17             [842, 853): 'Asian Games'     MISC   \n",
       "18              [875, 885): 'Uzbekistan'      LOC   \n",
       "19                   [982, 987): 'Japan'      LOC   \n",
       "20                 [1072, 1077): 'Syria'      LOC   \n",
       "21         [1079, 1092): 'Takuya Takagi'      PER   \n",
       "22  [1148, 1168): 'Hiroshige Yanagimoto'      PER   \n",
       "23                [1187, 1193): 'Syrian'     MISC   \n",
       "24           [1216, 1227): 'Salem Bitar'      PER   \n",
       "25                 [1328, 1333): 'Syria'      LOC   \n",
       "26          [1360, 1372): 'Hassan Abbas'      PER   \n",
       "27                 [1489, 1494): 'Bitar'      PER   \n",
       "28        [1503, 1517): 'Nader Jokhadar'      PER   \n",
       "29                 [1528, 1533): 'Syria'      LOC   \n",
       "30                 [1592, 1597): 'Japan'      LOC   \n",
       "31                [1621, 1627): 'Syrian'     MISC   \n",
       "32                [1686, 1692): 'Syrian'     MISC   \n",
       "33                 [1702, 1707): 'Bitar'      PER   \n",
       "34                 [1749, 1754): 'Japan'      LOC   \n",
       "35              [1761, 1769): 'Shu Kamo'      PER   \n",
       "36                [1784, 1790): 'Syrian'     MISC   \n",
       "37               [1825, 1832): 'Syrians'     MISC   \n",
       "38                 [1928, 1933): 'Japan'      LOC   \n",
       "39             [1951, 1960): 'World Cup'     MISC   \n",
       "40                  [2001, 2005): 'FIFA'      ORG   \n",
       "41                   [2056, 2059): 'UAE'      LOC   \n",
       "42                [2065, 2071): 'Kuwait'      LOC   \n",
       "43           [2076, 2087): 'South Korea'      LOC   \n",
       "44             [2096, 2105): 'Indonesia'      LOC   \n",
       "\n",
       "                                             sentence  sentence_id new_type  \n",
       "0   [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...           11      LOC  \n",
       "1   [11, 65): 'SOCCER- JAPAN GET LUCKY WIN, CHINA ...           11      PER  \n",
       "2                             [66, 77): 'Nadim Ladki'           66      PER  \n",
       "3   [78, 117): 'AL-AIN, United Arab Emirates 1996-...           78      LOC  \n",
       "4   [78, 117): 'AL-AIN, United Arab Emirates 1996-...           78      LOC  \n",
       "5   [118, 244): 'Japan began the defence of their ...          118      LOC  \n",
       "6   [118, 244): 'Japan began the defence of their ...          118     MISC  \n",
       "7   [118, 244): 'Japan began the defence of their ...          118      LOC  \n",
       "8   [245, 374): 'But China saw their luck desert t...          245      LOC  \n",
       "9   [245, 374): 'But China saw their luck desert t...          245      LOC  \n",
       "10  [375, 617): 'China controlled most of the matc...          375      LOC  \n",
       "11  [375, 617): 'China controlled most of the matc...          375     MISC  \n",
       "12  [375, 617): 'China controlled most of the matc...          375      PER  \n",
       "13  [375, 617): 'China controlled most of the matc...          375     MISC  \n",
       "14  [618, 735): 'Oleg Shatskiku made sure of the w...          618      PER  \n",
       "15  [736, 821): 'The former Soviet republic was pl...          736     MISC  \n",
       "16  [736, 821): 'The former Soviet republic was pl...          736     MISC  \n",
       "17  [822, 917): 'Despite winning the Asian Games t...          822     MISC  \n",
       "18  [822, 917): 'Despite winning the Asian Games t...          822      LOC  \n",
       "19  [918, 1078): 'Two goals from defensive errors ...          918      LOC  \n",
       "20  [918, 1078): 'Two goals from defensive errors ...          918      LOC  \n",
       "21  [1079, 1291): 'Takuya Takagi scored the winner...         1079      PER  \n",
       "22  [1079, 1291): 'Takuya Takagi scored the winner...         1079      PER  \n",
       "23  [1079, 1291): 'Takuya Takagi scored the winner...         1079     MISC  \n",
       "24  [1079, 1291): 'Takuya Takagi scored the winner...         1079      PER  \n",
       "25  [1292, 1350): 'It was the second costly blunde...         1292      LOC  \n",
       "26  [1351, 1502): 'Defender Hassan Abbas rose to i...         1351      PER  \n",
       "27  [1351, 1502): 'Defender Hassan Abbas rose to i...         1351      PER  \n",
       "28  [1503, 1591): 'Nader Jokhadar had given Syria ...         1503      PER  \n",
       "29  [1503, 1591): 'Nader Jokhadar had given Syria ...         1503      LOC  \n",
       "30  [1592, 1701): 'Japan then laid siege to the Sy...         1592      LOC  \n",
       "31  [1592, 1701): 'Japan then laid siege to the Sy...         1592     MISC  \n",
       "32  [1592, 1701): 'Japan then laid siege to the Sy...         1592     MISC  \n",
       "33  [1702, 1748): 'Bitar pulled off fine saves whe...         1702      PER  \n",
       "34  [1749, 1820): 'Japan coach Shu Kamo said: ' ' ...         1749      LOC  \n",
       "35  [1749, 1820): 'Japan coach Shu Kamo said: ' ' ...         1749      PER  \n",
       "36  [1749, 1820): 'Japan coach Shu Kamo said: ' ' ...         1749     MISC  \n",
       "37  [1821, 1925): 'The Syrians scored early and th...         1821     MISC  \n",
       "38  [1928, 2049): 'Japan, co-hosts of the World Cu...         1928      LOC  \n",
       "39  [1928, 2049): 'Japan, co-hosts of the World Cu...         1928     MISC  \n",
       "40  [1928, 2049): 'Japan, co-hosts of the World Cu...         1928      ORG  \n",
       "41  [2050, 2137): 'Hosts UAE play Kuwait and South...         2050      LOC  \n",
       "42  [2050, 2137): 'Hosts UAE play Kuwait and South...         2050      LOC  \n",
       "43  [2050, 2137): 'Hosts UAE play Kuwait and South...         2050      LOC  \n",
       "44  [2050, 2137): 'Hosts UAE play Kuwait and South...         2050      LOC  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corrected_entities = entity_mentions.copy(True)\n",
    "new_types = corrected_entities[\"ent_type\"].copy()\n",
    "new_types[widget.selected] = \"ORG\"\n",
    "corrected_entities[\"new_type\"] = new_types\n",
    "corrected_entities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d826972-07ef-40dd-9d60-534fe1198c1b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}