![\"코랩에서](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
\n",
"| \n",
"\n",
" |
![\"Marie](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_marie-curie.png?raw=1\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "I2DxvBM0z41d"
},
"source": [
"## 리뷰 기반 QA 시스템 구축하기"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1-rCKMXJz41e"
},
"source": [
"### 데이터셋"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QhStaRsGz41f"
},
"source": [
"![\"Phone](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_phone.png?raw=1\")
"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 99,
"referenced_widgets": [
"dd6eefee0b604dc4ad26235d6650f819",
"88e6e4bb9bad4c778884761f73abe971",
"b527fdc36a964d3091a759631ad4c076",
"747e599ed26444568eb963ae9931b639",
"f19540eca4bb4b1b98d4c714ca9575be",
"f6823486a3a84c50b59f1e76d8017f99",
"441787b1af6145ae83d72f058a160a2b",
"18abb84c6a4145f0b1bf4fa2dae96b5f",
"1381dfaa69d943deab27ca802b39910b",
"3969b4ea396e495691294eeee8c1a8ae",
"8fdb8370e9ec4307883a1b72c5eb694f",
"e3453b02063b4e4c943f685731456e71",
"b185a07760f84dcbba6eb01fb92c8b23",
"2b2f3f2f7be04309903bc5e211011801",
"7a6e5f51e7874b4f8b18afc1392315ef",
"e241b3f75d4f4af2989f946643babcf6",
"3cddbdc3dd13443b85a41fe1ccf9f76f",
"327a6d88210b492fa281630eb7b2bf08",
"8a621c01ed7e4072a6c9733792df4669",
"2696479e59804459b9501732cb11f42a",
"22e46c837abd4f91bdad7aa2077d1842",
"a758c96d91dd4adbb308bf9a342f0c8b"
]
},
"id": "tHcoy_Rtz41f",
"outputId": "2543c957-c065-42d1-d4e6-29eb675a9bf1"
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "dd6eefee0b604dc4ad26235d6650f819",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Downloading: 0%| | 0.00/2.65k [00:00\n",
" | \n", " | title | \n", "question | \n", "answers.text | \n", "answers.answer_start | \n", "context | \n", "
|---|---|---|---|---|---|
| 791 | \n", "B005DKZTMG | \n", "Does the keyboard lightweight? | \n", "[this keyboard is compact] | \n", "[215] | \n", "I really like this keyboard. I give it 4 stars because it doesn't have a CA... | \n", "
| 1159 | \n", "B00AAIPT76 | \n", "How is the battery? | \n", "[] | \n", "[] | \n", "I bought this after the first spare gopro battery I bought wouldn't hold a c... | \n", "
![\"SQuAD](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_squad-sota.png?raw=1\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "V56fOUQRz41l"
},
"source": [
"### 사이드바 끝"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WWtK_6STz41m"
},
"source": [
"### 텍스트에서 답 추출하기"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JKaueCUUz41m"
},
"source": [
"#### 범위 분류"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "w7FgPqwtz41m"
},
"source": [
"![\"QA](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_qa-head.png?raw=1\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VGWPDO8Dz41n"
},
"source": [
"![\"SQuAD](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_squad-models.png?raw=1\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4Wj8cnXJz41n"
},
"source": [
"#### QA를 위한 텍스트 토큰화"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 145,
"referenced_widgets": [
"a5aa206de3b243ec9ed2a6e914a448fa",
"b8536ec5c895460ebfa32a103fd80591",
"6d5046a2cb704b3ca5758c10a86288bf",
"5307c50f07cd421db587496c39e4d175",
"6d9e00830be4431bb2bbeeff660dd6eb",
"b5afa237918a4f7e918730dd78f23724",
"1d8743ebb1664e818055d55f846725fa",
"00c99c2a8fd94b9a8d992fdcf11f6e61",
"50939b3e6f914edc94c7568b5cc47e22",
"58aa92721d354129a8d31ae34f2cf855",
"a86b2402ffbc46fd9d92325cf1cfe3aa",
"06e3d8ab6e374e5ba9f4c4f0f0424b04",
"7d0fed82d1db4d558c67c75c8378b412",
"fb64a118ff924db68c44208bb1e1f7bb",
"7f635a3e72cf4298b1ff4dbdfbb634c3",
"5c5590848528421ab7c081610f8dfc58",
"cb8a67217c974ad9a04690da613cdda7",
"066df9e253f042e2ae242fa1d5a03596",
"9628c68b4f2041a8ae8e56a59930359f",
"5940bb21f0cf496f9ddb7b40c3b6e353",
"64224a316b1348a1803cd793e4f27cb7",
"4c326dcae0ca4ebc8b19008ceae6a93c",
"f709228b2006482da1050f30604bbb11",
"6d606e63dae34cc9820a345f969b4ff4",
"daf33cfaaa604cdaa44d60690068555f",
"1fa630961cdb4f83a96b08e3db2e1ce3",
"5fbd5d0edc6d41678a5c3c155cde5279",
"fc1aa6d56f004f6fb2265225c8517f65",
"3079624f39d3437489e46aa5e7783301",
"847cb3ec626e4c18b3fd93af01736368",
"5bb6c9761c034d50b967c22edbddb736",
"268e2d65ee0d4675b364e514cc6039a1",
"f2b73e98882e477daff3a56798012368",
"e20d8ade0bd347d39f0eaedffbd10ba0",
"018a5325476c45b28613895fd0df9d0d",
"ac1d74729df34a1fac1f530c3c6b39cb",
"f4d64cf610cd4c3fb62124908a8c791d",
"b5893539ee084860ae1cdda6debd8dd6",
"0edd8fc74cb34fcd9a92480b87bd8244",
"687a73a209a942a5b1bba2e0490f7c1d",
"6cdab1e4b4204a8286d4c210b76144b1",
"d997fc3e56a3461790bee7b7a589d95d",
"b63d8d35559c47099546d9ebfce22783",
"58531126fa0f48f090de938365ee83bd"
]
},
"id": "9qGdwgLQz41o",
"outputId": "254517bb-3b5d-4e18-98cc-5c32befe555c"
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "a5aa206de3b243ec9ed2a6e914a448fa",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Downloading: 0%| | 0.00/477 [00:00\n",
" | \n", " | 0 | \n", "1 | \n", "2 | \n", "3 | \n", "4 | \n", "5 | \n", "6 | \n", "7 | \n", "8 | \n", "9 | \n", "... | \n", "18 | \n", "19 | \n", "20 | \n", "21 | \n", "22 | \n", "23 | \n", "24 | \n", "25 | \n", "26 | \n", "27 | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| input_ids | \n", "101 | \n", "2129 | \n", "2172 | \n", "2189 | \n", "2064 | \n", "2023 | \n", "2907 | \n", "1029 | \n", "102 | \n", "2019 | \n", "... | \n", "2061 | \n", "2055 | \n", "25961 | \n", "2847 | \n", "5834 | \n", "2006 | \n", "5371 | \n", "2946 | \n", "1012 | \n", "102 | \n", "
| token_type_ids | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "1 | \n", "... | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "
| attention_mask | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "... | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "
3 rows × 28 columns
\n", "![\"Sliding](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_sliding-window.png?raw=1\")
"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8emrQ8U0z41u"
},
"outputs": [],
"source": [
"example = dfs[\"train\"].iloc[0][[\"question\", \"context\"]]\n",
"tokenized_example = tokenizer(example[\"question\"], example[\"context\"],\n",
" return_overflowing_tokens=True, max_length=100,\n",
" stride=25)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "IcGeOmO6z41u",
"outputId": "783c9a4a-84a1-42e6-cdb6-fdcdf4f39924"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"#0 윈도에는 100개의 토큰이 있습니다.\n",
"#1 윈도에는 88개의 토큰이 있습니다.\n"
]
}
],
"source": [
"for idx, window in enumerate(tokenized_example[\"input_ids\"]):\n",
" print(f\"#{idx} 윈도에는 {len(window)}개의 토큰이 있습니다.\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "zOFXgi7Bz41v",
"outputId": "dd702de4-f9e5-4a2e-f763-c3d5b4c45aeb"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[CLS] how is the bass? [SEP] i have had koss headphones in the past, pro 4aa and\n",
"qz - 99. the koss portapro is portable and has great bass response. the work\n",
"great with my android phone and can be \" rolled up \" to be carried in my\n",
"motorcycle jacket or computer bag without getting crunched. they are very light\n",
"and don't feel heavy or bear down on your ears even after listening to music\n",
"with them on all day. the sound is [SEP]\n",
"\n",
"[CLS] how is the bass? [SEP] and don't feel heavy or bear down on your ears even\n",
"after listening to music with them on all day. the sound is night and day better\n",
"than any ear - bud could be and are almost as good as the pro 4aa. they are \"\n",
"open air \" headphones so you cannot match the bass to the sealed types, but it\n",
"comes close. for $ 32, you cannot go wrong. [SEP]\n",
"\n"
]
}
],
"source": [
"for window in tokenized_example[\"input_ids\"]:\n",
" print(f\"{tokenizer.decode(window)} \\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "M50OhvNGz41v"
},
"source": [
"### 헤이스택을 사용해 QA 파이프라인 구축하기"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2IWPWQDaz41v"
},
"source": [
"![\"QA](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_retriever-reader.png?raw=1\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PH0ytdBNz41v"
},
"source": [
"#### 문서 저장소 초기화하기"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "INnrsUNLz41v"
},
"outputs": [],
"source": [
"url = \"\"\"https://artifacts.elastic.co/downloads/elasticsearch/\\\n",
"elasticsearch-7.9.3-linux-x86_64.tar.gz\"\"\"\n",
"!wget -nc -q {url}\n",
"!tar -xzf elasticsearch-7.9.3-linux-x86_64.tar.gz"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "T1zkGdtPz41w"
},
"outputs": [],
"source": [
"import os\n",
"from subprocess import Popen, PIPE, STDOUT\n",
"\n",
"# 백그라운드 프로세스로 일래스틱서치를 실행합니다\n",
"!chown -R daemon:daemon elasticsearch-7.9.3\n",
"es_server = Popen(args=['elasticsearch-7.9.3/bin/elasticsearch'],\n",
" stdout=PIPE, stderr=STDOUT, preexec_fn=lambda: os.setuid(1))\n",
"# 일래스틱서치가 시작할 때까지 기다립니다\n",
"!sleep 30"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "wL-1x19Nz41w",
"outputId": "937d5965-c4f2-4c21-8059-d2121fb102e3"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:haystack.utils:Tried to start Elasticsearch through Docker but this failed. It is likely that there is already an existing Elasticsearch instance running. \n"
]
}
],
"source": [
"# 또는 도커가 설치되어 있다면\n",
"from haystack.utils import launch_es\n",
"\n",
"launch_es()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "wl5rLHzOz41w",
"outputId": "db87b213-f253-4f39-f470-ed040cd4d7dd"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"name\" : \"5e41be2d74e7\",\n",
" \"cluster_name\" : \"elasticsearch\",\n",
" \"cluster_uuid\" : \"OG6mo2jGQWmDRmm-VOOAHA\",\n",
" \"version\" : {\n",
" \"number\" : \"7.9.3\",\n",
" \"build_flavor\" : \"default\",\n",
" \"build_type\" : \"tar\",\n",
" \"build_hash\" : \"c4138e51121ef06a6404866cddc601906fe5c868\",\n",
" \"build_date\" : \"2020-10-16T10:36:16.141335Z\",\n",
" \"build_snapshot\" : false,\n",
" \"lucene_version\" : \"8.6.2\",\n",
" \"minimum_wire_compatibility_version\" : \"6.8.0\",\n",
" \"minimum_index_compatibility_version\" : \"6.0.0-beta1\"\n",
" },\n",
" \"tagline\" : \"You Know, for Search\"\n",
"}\n"
]
}
],
"source": [
"!curl -X GET \"localhost:9200/?pretty\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AzpVTzauz41x"
},
"outputs": [],
"source": [
"from haystack.document_store.elasticsearch import ElasticsearchDocumentStore\n",
"\n",
"# 밀집 리트리버에서 사용할 문서 임베딩을 반환합니다.\n",
"document_store = ElasticsearchDocumentStore(return_embedding=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ity-bUF7z41x"
},
"outputs": [],
"source": [
"# 노트북을 다시 시작할 때 일래스틱서치 저장소를 모두 비우는 것이 좋습니다.\n",
"if len(document_store.get_all_documents()) or len(document_store.get_all_labels()) > 0:\n",
" document_store.delete_documents(\"document\")\n",
" document_store.delete_documents(\"label\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "qbyi1qD-z41x",
"outputId": "9a19509e-1ef6-483f-9a41-e47f5e341f44"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1615개 문서가 저장되었습니다\n"
]
}
],
"source": [
"for split, df in dfs.items():\n",
" # 중복 리뷰를 제외시킵니다\n",
" docs = [{\"text\": row[\"context\"],\n",
" \"meta\":{\"item_id\": row[\"title\"], \"question_id\": row[\"id\"],\n",
" \"split\": split}}\n",
" for _,row in df.drop_duplicates(subset=\"context\").iterrows()]\n",
" document_store.write_documents(docs, index=\"document\")\n",
"\n",
"print(f\"{document_store.get_document_count()}개 문서가 저장되었습니다\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BRVKygWTz41y"
},
"source": [
"#### 리트리버 초기화하기"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "G-BmiSOGz41y"
},
"outputs": [],
"source": [
"from haystack.retriever.sparse import ElasticsearchRetriever\n",
"\n",
"es_retriever = ElasticsearchRetriever(document_store=document_store)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fdnSMdOIz41y"
},
"outputs": [],
"source": [
"item_id = \"B0074BW614\"\n",
"query = \"Is it good for reading?\"\n",
"retrieved_docs = es_retriever.retrieve(\n",
" query=query, top_k=3, filters={\"item_id\":[item_id], \"split\":[\"train\"]})"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "HrUYAVVGz410",
"outputId": "36bc13d7-384f-4a37-edd9-1e7a1cde0b9c"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'text': 'This is a gift to myself. I have been a kindle user for 4 years and\n",
"this is my third one. I never thought I would want a fire for I mainly use it\n",
"for book reading. I decided to try the fire for when I travel I take my laptop,\n",
"my phone and my iPod classic. I love my iPod but watching movies on the plane\n",
"with it can be challenging because it is so small. Laptops battery life is not\n",
"as good as the Kindle. So the Fire combines for me what I needed all three to\n",
"do. So far so good.', 'score': 6.243799, 'probability': 0.6857824513476455,\n",
"'question': None, 'meta': {'item_id': 'B0074BW614', 'question_id':\n",
"'868e311275e26dbafe5af70774a300f3', 'split': 'train'}, 'embedding': None, 'id':\n",
"'252e83e25d52df7311d597dc89eef9f6'}\n"
]
}
],
"source": [
"print(retrieved_docs[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-Lvpd41rz411"
},
"source": [
"#### 리더 초기화하기"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ddfnri_bz411"
},
"outputs": [],
"source": [
"from haystack.reader.farm import FARMReader\n",
"\n",
"model_ckpt = \"deepset/minilm-uncased-squad2\"\n",
"max_seq_length, doc_stride = 384, 128\n",
"reader = FARMReader(model_name_or_path=model_ckpt, progress_bar=False,\n",
" max_seq_len=max_seq_length, doc_stride=doc_stride,\n",
" return_no_answer=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "qXMSYzsXz411",
"outputId": "4172f189-1b62-4824-e26b-41b24c7b3ffa"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'query': 'How much music can this hold?', 'no_ans_gap': 12.64809501171112,\n",
"'answers': [{'answer': '6000 hours', 'score': 10.699626922607422, 'probability':\n",
"0.3988155424594879, 'context': 'An MP3 is about 1 MB/minute, so about 6000 hours\n",
"depending on file size.', 'offset_start': 38, 'offset_end': 48,\n",
"'offset_start_in_doc': 38, 'offset_end_in_doc': 48, 'document_id':\n",
"'e344757014e804eff50faa3ecf1c9c75'}]}\n"
]
}
],
"source": [
"print(reader.predict_on_texts(question=question, texts=[context], top_k=1))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "P91oPhTqz411"
},
"source": [
"#### 모두 합치기"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UB2q1x6Jz412"
},
"outputs": [],
"source": [
"from haystack.pipeline import ExtractiveQAPipeline\n",
"\n",
"pipe = ExtractiveQAPipeline(reader, es_retriever)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "yVLnBu9Rz41_",
"outputId": "5cd2bffa-2f43-4ba7-f295-f0cfc06881cd"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"질문: Is it good for reading?\n",
"\n",
"답변 1: I mainly use it for book reading\n",
"해당 리뷰 텍스트: ... is my third one. I never thought I would want a fire for I\n",
"mainly use it for book reading. I decided to try the fire for when I travel I\n",
"take my la...\n",
"\n",
"\n",
"\n",
"답변 2: the larger screen compared to the Kindle makes for easier reading\n",
"해당 리뷰 텍스트: ...ght enough that I can hold it to read, but the larger screen\n",
"compared to the Kindle makes for easier reading. I love the color, something I\n",
"never thou...\n",
"\n",
"\n",
"\n",
"답변 3: it is great for reading books when no light is available\n",
"해당 리뷰 텍스트: ...ecoming addicted to hers! Our son LOVES it and it is great for\n",
"reading books when no light is available. Amazing sound but I suggest good\n",
"headphones t...\n",
"\n",
"\n",
"\n"
]
}
],
"source": [
"n_answers = 3\n",
"preds = pipe.run(query=query, top_k_retriever=3, top_k_reader=n_answers,\n",
" filters={\"item_id\": [item_id], \"split\":[\"train\"]})\n",
"\n",
"print(f\"질문: {preds['query']} \\n\")\n",
"for idx in range(n_answers):\n",
" print(f\"답변 {idx+1}: {preds['answers'][idx]['answer']}\")\n",
" print(f\"해당 리뷰 텍스트: ...{preds['answers'][idx]['context']}...\")\n",
" print(\"\\n\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XtWniguTz42A"
},
"source": [
"## QA 파이프라인 개선하기"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lVGUcNKgz42A"
},
"source": [
"### 리트리버 평가하기"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "gYCgemwjz42A"
},
"outputs": [],
"source": [
"from haystack.pipeline import Pipeline\n",
"from haystack.eval import EvalDocuments\n",
"\n",
"class EvalRetrieverPipeline:\n",
" def __init__(self, retriever):\n",
" self.retriever = retriever\n",
" self.eval_retriever = EvalDocuments()\n",
" pipe = Pipeline()\n",
" pipe.add_node(component=self.retriever, name=\"ESRetriever\",\n",
" inputs=[\"Query\"])\n",
" pipe.add_node(component=self.eval_retriever, name=\"EvalRetriever\",\n",
" inputs=[\"ESRetriever\"])\n",
" self.pipeline = pipe\n",
"\n",
"\n",
"pipe = EvalRetrieverPipeline(es_retriever)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "amX_CSHdz42A"
},
"outputs": [],
"source": [
"from haystack import Label\n",
"\n",
"labels = []\n",
"for i, row in dfs[\"test\"].iterrows():\n",
" # 리트리버에서 필터링을 위해 사용하는 메타데이터\n",
" meta = {\"item_id\": row[\"title\"], \"question_id\": row[\"id\"]}\n",
" # 답이 있는 질문을 레이블에 추가합니다\n",
" if len(row[\"answers.text\"]):\n",
" for answer in row[\"answers.text\"]:\n",
" label = Label(\n",
" question=row[\"question\"], answer=answer, id=i, origin=row[\"id\"],\n",
" meta=meta, is_correct_answer=True, is_correct_document=True,\n",
" no_answer=False)\n",
" labels.append(label)\n",
" # 답이 없는 질문을 레이블에 추가합니다\n",
" else:\n",
" label = Label(\n",
" question=row[\"question\"], answer=\"\", id=i, origin=row[\"id\"],\n",
" meta=meta, is_correct_answer=True, is_correct_document=True,\n",
" no_answer=True)\n",
" labels.append(label)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "FrhGuFugz42B",
"outputId": "9ff252fa-948f-4e96-e846-7fa26ee0784c"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'id': '336690e2-4398-4ba8-9744-b6363b373427', 'created_at': None, 'updated_at':\n",
"None, 'question': 'What is the tonal balance of these headphones?', 'answer': 'I\n",
"have been a headphone fanatic for thirty years', 'is_correct_answer': True,\n",
"'is_correct_document': True, 'origin': 'd0781d13200014aa25860e44da9d5ea7',\n",
"'document_id': None, 'offset_start_in_doc': None, 'no_answer': False,\n",
"'model_id': None, 'meta': {'item_id': 'B00001WRSJ', 'question_id':\n",
"'d0781d13200014aa25860e44da9d5ea7'}}\n"
]
}
],
"source": [
"print(labels[0])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "C8eWcYU5z42B",
"outputId": "64d47f55-8bd8-47f6-9b43-6c647b8d1aba"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"358개의 질문 답변 쌍을 로드했습니다.\n"
]
}
],
"source": [
"document_store.write_labels(labels, index=\"label\")\n",
"print(f\"\"\"{document_store.get_label_count(index=\"label\")}개의 \\\n",
"질문 답변 쌍을 로드했습니다.\"\"\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "8_ezogm9z42B",
"outputId": "adbe2e45-8cea-4e6d-c100-c089fa06f3b6"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"330\n"
]
}
],
"source": [
"labels_agg = document_store.get_all_labels_aggregated(\n",
" index=\"label\",\n",
" open_domain=True,\n",
" aggregate_by_meta=[\"item_id\"]\n",
")\n",
"print(len(labels_agg))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-IBkcanfz42C"
},
"outputs": [],
"source": [
"def run_pipeline(pipeline, top_k_retriever=10, top_k_reader=4):\n",
" for l in labels_agg:\n",
" _ = pipeline.pipeline.run(\n",
" query=l.question,\n",
" top_k_retriever=top_k_retriever,\n",
" top_k_reader=top_k_reader,\n",
" top_k_eval_documents=top_k_retriever,\n",
" labels=l,\n",
" filters={\"item_id\": [l.meta[\"item_id\"]], \"split\": [\"test\"]})"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Q7SeC8XWz42C",
"outputId": "6878d3b0-71ed-47ee-fc00-a5ebdc5d5ad0"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"재현율@3: 0.95\n"
]
}
],
"source": [
"run_pipeline(pipe, top_k_retriever=3)\n",
"print(f\"재현율@3: {pipe.eval_retriever.recall:.2f}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UPZC7be_z42C"
},
"outputs": [],
"source": [
"def evaluate_retriever(retriever, topk_values = [1,3,5,10,20]):\n",
" topk_results = {}\n",
"\n",
" for topk in topk_values:\n",
" # 파이프라인을 만듭니다\n",
" p = EvalRetrieverPipeline(retriever)\n",
" # 테스트 세트에 있는 질문-답변 쌍을 반복합니다\n",
" run_pipeline(p, top_k_retriever=topk)\n",
" # 재현율을 저장합니다\n",
" topk_results[topk] = {\"recall\": p.eval_retriever.recall}\n",
"\n",
" return pd.DataFrame.from_dict(topk_results, orient=\"index\")\n",
"\n",
"\n",
"es_topk_df = evaluate_retriever(es_retriever)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 376
},
"id": "eGkM8RXLz42C",
"outputId": "a0b809c7-47f1-495c-ffac-263d617d7d08"
},
"outputs": [
{
"data": {
"application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1BhZ2VzIDIgMCBSIC9UeXBlIC9DYXRhbG9nID4+CmVuZG9iago4IDAgb2JqCjw8IC9FeHRHU3RhdGUgNCAwIFIgL0ZvbnQgMyAwIFIgL1BhdHRlcm4gNSAwIFIKL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gL1NoYWRpbmcgNiAwIFIKL1hPYmplY3QgNyAwIFIgPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9Bbm5vdHMgMTAgMCBSIC9Db250ZW50cyA5IDAgUiAvTWVkaWFCb3ggWyAwIDAgMzk4LjU1OTM3NSAyNjYuMDkgXQovUGFyZW50IDIgMCBSIC9SZXNvdXJjZXMgOCAwIFIgL1R5cGUgL1BhZ2UgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0ZpbHRlciAvRmxhdGVEZWNvZGUgL0xlbmd0aCAxMiAwIFIgPj4Kc3RyZWFtCnicjVbLbtswENSZX8FjcwjN5fJ5jPsIEKCHJAZ6KHooXMdt4qRNAzS/36GkSKQfsm04scbkzOxytavZh9W/X8vVzeVcvr8Vs/Fq+SJI3ovZBcn1i9TyHp9XSfJS1os08EfBKSrnEgeHy015abxXOgHT5de86KcQT9BY4+ISxGshnH/bZUlZ1/KS4hLalJBxUcVMOG4cILDfwb3p3K8hgAhURAxZDogIpEIIzoVBbUQ4qJTFxBwOX8Uz/mp5rsHi8y/BWMsUDeQUGyeXj2K+kLNPJMnIxV0b4OKH+CrfNXQmv8nFlfi4ENeiNSBIs4rkIttBuYAOS6ekUnLOUGRNJ0jzHml2KjpLWDtIj9BhaWKjNHNIia05RdvtahtyKtkUtR+0C+iwttFRsU6EVeakjDd6V5yDV4QgE4+1NUKHxdmjkrV2PhLeJ6ibWr2Mw2hlo4kEgqTCFMnDbgBjifuEdHCMFjE4Vnob3ZtDoyJkvVXBEkdrUpiMQjeqSfvyONogHEsEFVc2RnTCBmmrTHApu0A1HfVhJn1YlEdoA6qMjPCUE5vvO23hQ3t/3ImddBJRRroNqXIywlNOIiu2TmcfED/qxE85MSYoa9uQSicFPOHE5P8hOfggN1mmnZNY13tmOs+c5Nt7hcjCVlf2E1yLM2xIKAsduX0B+938ac6bh0Y2N2fS2vrHVbNsvjebZlPJy+0ZwmyRHEM5cvl3Jb/IJ2nklSTl2lmAexKtHJXibMJF6L5pI262J10xHhxGELNnX7duCkEhTFAgv0VfNUzonkhnznvR84xNSmvQ5Ju2aEcVLm7FtXxuPethhtXOdufmnoEInp1p+rh3mmLlSZO4WtfvPsg4u+BuFF/lWY/PaxtKP/mZtPLoGpk99ILRoCmnEUKKoh1stRBHN5iKytRAQvrlUhQQnj9M6IhGLKDY88aBGh3ft8ig3yPL0maPgUvjsCqM0QNNj/TsBdJ5ANeIdVY3FdSFM3L3AW/naZkfdOZvDzptYaNIjhW33ClurHpTwqGl9gAw7CoIBjEotyAcrW5Lsz/T/uGqHKHWv3l2mD+TXWDefMYErR4ersV/GO4dpAplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjgxNgplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagoxNyAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDIwIDAgUiAvTGVuZ3RoMSA4MzE2ID4+CnN0cmVhbQp4nNVZe1xVVfZf+6yz7wsunHu5FwQul8vjgoYPAvGV6Y3UTM1IyREbGq4Cki9QfKRk4As0NcQHlqndfFVkDpmjoFaWlBry+zXjY6apxkxLK3w0Q9mPYN/fOgdwrPnNb36/v36f3zl3nbPXPmvvvb5r7b32OucCAwArXWRwPTBs+AgIgQgA1pNqQx/IeHh87OMJkcQPIxr7wPhH0xN3DT0OIG2k59UP3Zc50jg0dTkAqvI/PDy+T8r0H3+7iDqLJ37ClJneImmeOZf4p6j9yCnz57rgiaiBALyBeJFfNHXm7L7zpwHoiYfXpnqLi0BPJxiKiA+cOmNhPntp9nTiqX1wRkGeN9eQ/V45QNhpet6vgCrML+orALrJxMcXzJz7ZHGcuS/x3YkvmFE4xZtUlXiE+HWq/Ezvk0VyhW428SeJd83yzsxLvHbvUeKvkj5niwqL57KNUi+AiKOq/kVz8oru0X9HxUgb6VwAqq0CoeOQiEMI1+pUMkFvGAzSsBFjMiFohnfuLOgGqlbg9wPcLmnS0/PmzAJDZztGzyTtbqB7jSZ5FM6TFQI6BvIfIhoGXUcC/JPD/7oqSffp/jN03dVJw/xmrfZT8brYJahGpX91+D/tHFftMUHlfnEw0k+GYLCAFewQSlgZcSYNkUR2AQ0zp6tOQ2inq/RfDISdcjrqT7WIkfrosud62Ag2zZ6LvHO8k2G5d87MWbB88hzvE7B8indWMV0L8ubQdeGcGbB8al4hlafOyZsOywu8s0imIG8y1Uz3zvLC8hneQpd6Jb8sn+mdWwDLZ01XawqnemfC8jnzZpHk3PxZU+laoPZ/h+/uRK3q9TlchQwYHAj6z9RKaSlByKH5ca5DSC3feXTxrKCz/Oi/tj8Lon7n/2s5mNQhq95vl+/o42f1k+7g5/y9LA0A+KmSyoNvN+1OKE23EdNVDmLrVG/yVP48sc6OO/4R8iUr9RCgQzTIkiTfbtF5ZOQPzwUPuKBIZxM2tkU/k13KAea/0GVX7CQHgDbiVuKYxstQpt0VqgmkHnpDPxgI6fAQjINM8MIUyIMZUKR5yAW9IBkG/OLpdCj0+/2X/H/0n/ef8zf5j/vf8b/t3+9/w1/rf83/6s81/YejY53v7eTM2igdpOpLMYI06ljXyUT9QJ3pQDoAadkhn95JwUQPdZJCNK6TLESZnURWJK07KIRoSidR3CEkHWQnolhIqCk2ExUSqXGyG0uDOmik812KH1vZHuLyqX421fik/bAC5lHNcdbIVlFsa4Q9cBPOkGQFNGKNDGwUpFItwMdcghaWCQeoj4HMxgbqdTLIY+UD8ji5Tr4iN0F/uVhuknPkYpaKO/gEvodoIL5Pc+AUREMduwDFcBi/xlQ8Kg+Tg+ACNmENfEmjqHZqhErYBSWki40VQqlUIo2jmhO8CbbQWUjPm9h2doa0O8yWwTl4DmVpJGxn5whXI/wAyzBTKiXzp0r5pP8J6quJ2m+BYgog55gJhJREdaQ9jTVZu0ZhL35OO29CKY2cCbt0dTqbPo5GUS22hx1nzboN4IMz+GucjZ+wFXKc/Io8Eio7LIA5UEl9b1Hb6PLZQsKuniVq79ICOYfVwNdyjn4y9f2+iojGPCCNI0T5FMPzYYFOIUz3sBW4ijRVn0ZBk36U3IfaUw/6xZoPCzENplGpBPbBfuiF1VBJPWl4df35D9Ryq3yRMFeytdIP0ITDoAfky9fJ1ur0qAY4pNdxGSUGPV1KreR+MLfW88hE18msmF49f8G6FL2rFjJqzQtddX5/xkQ5kmfVckctug21sjvu4j97eLFXz9EZE1217cOHdfY6PGcY1Y2fSEWVo2qqHz5Me6YOWsvd9Hswp9Y1pcD1jPJM3KBnlLxB6rKhfYB2Olq36krPZRekUmkZlS0HYask016hfHr6blDONieHxNhjcqXI9i+lZbvUuPIJXfaBUGUPwTKJhdO2r7Q13w19mpP7p9rjPjlzRgjqvcJ/Sa4kjwdAGMR5QnQ+K/gCq6xruhkdwU502CO7UasWaqdcbmlWriezWMmiWFNTrBZFSkwBiwJxsepVWr112zb6bdv2EzOKWz/9JG4xI88QTeI0URNLpbMvS/WJYlEuKkQxW8sWskVsrarrRQoNk2hHMIHHY09Hnyz5+BI9+IyGaJ0DIZoFKGdH1wZnTqwnYc+ArOaGDiApLc2EnOyaFcsOBGOwLGX3j7HwNHeqhawh2CjxPMv7kI1q21UjF4+sG9l6roY6oDkgjyLEDtjuSQyPiMRuDguXwcK5nK68ZNlo9tmqZIoFoJgkZnKEKaiLUtpG19ozR9eGZj42utaW+Rhpgv5jA7IazjYfO2axDuzUpkXTRq/wa3p+jdU6FEvYQNLNk/KoPIFP0C+SF/H5kRXheooU4XIETRnHXJivmxdRHDnXsRTKw5dGLI1c6ngFXom0ZEO2m0Ck9YP+Q1ha34S4WJ0+bQhLTZHtNp2e8oRV0rttY8iMqd6HXi7/zZknF52deJXZhj8WLlpqamoWsKpBMzc/uKA6/f7Td6dcfe/Xu4uixLeEfiv5u5jQd4ciT2+wh5jKjdHlrhCf3ewzbtA5fK4NcVW6NfadPUIdIYC2cEeCS3GgLdqo66EaITSzC79Rw08GaGkmlGQBpflyy+Vm5avrinaSVZKZx5jr9EZ7XbkxMmQzJ7Pb5JjYhMQ0JwHpR6iSWFpH4WfwcGjVTvGRuPr4iWmZJ2e+faJ+976Dm7bvfG7823OKT2V9xQKfRXd0w7rP/up2H787pbpy+aY9C4qKS+ITDrhcv9//1GvqqlHz6F00pyTaXZZ4opgZzYBoTgcM0Ps4wyVGFmgCh84gBwYpn46uDSBgZg1YoArs7OCG5hSL6tfLZwc3pxAWzbHyKXLuKdWldwXAXTASsuAJWADPgD6UJUECS8J+bCx7OPBh8wSWz+axRbiCmcmVRhaDqRZadpY4S0wa6oTERJo4d+5U++Pc3XYJm9pSXxE+lnOcPLSdPJRLmkfB4544OUJvKVeiInx6m09ZZZZ8sMS8Rr/LGeZgJnSASdE5lTZ2p18UVf3O1aKoq4VcpDRcVxewuoLJPaKhwztqyLCoNge7DX7mFtUbn2F4u6/nxJ6tLF6cFTceP14w6dj01z/88PVHXsrk52rE+uBgcf2b78T3Llfj3ckHt249GJ9A1q4k7au1eBIPEz3xITowlweCL1Tnc4TuVnyBq2KrHGvcgbFGR7gzxIEx0ZFuCjA0iS5rIeZy2+W/Tx+PjfZp1iQ1YZPcyBt1hHu/U8pm2SxWZ7eFdujK7L1ZXKyEXUDiXGo4ikkJlXatfPHFlUTMOOaFMSfPBN+zf/pFxsXNL0S7uM4yWOSYF/CewzteOnLkpR2HpYV18Qnir+LGr7LFjW+/Et9oAWoy2+1Uc5jVhGq1hioO+sD9Hnc3wpSo8zl7+axVzjWJO5O7Bcbf5bDHO4KNFDMpcAbHRCYrbQ3NLQ3NGpyuFaJxA2lp3AHB3ZtWeHxqSqi6tLVFEhcbn9a3X0iXAPlDWr1u9+516/bsFruXVoH/LxdE1ZL1O8WtW7fErV0jq5Yt3bBh6bIq6f0tFRVbXiiv2DLBtb/szY8+erNsvyv2g8qPr179uPID5p27dOlcIvLTEkJUQYi6aX6K00eHs3II95l2yz5YFRrtU6pC17j1DkdMiBNiYx1mzU2kftdO8JX4vstLoQ3h70UcizzmOBb1nrMhWl9jPWr92orkp/7ajLKGBJGHIK0vpHb4JjaBdcEiG1wcs3U0eWfQ/hmfi5+Y8gVDZhFviC/HbGVDOj0YTb5hZmad8GsW/O1XLFTbRF4UjzmlzV3+U710itZ8M60cA2WLLvJSXDhsMhk3WZewTabXoy0BBikkPJpDkCOUhzt6G8FhlWMIVUobbSNqDNPCeLPqooHJ+4NjGU00S0yHprcL7pg73RTDNrBhO7dt2ymOsqSNVVUbRYAkX2kte2rTbnHzp/ar0qn2zypWr1kh5YshhXNmF+059saqHTZX43Mn/0wrnfTVRZO+gfCOpy9a9Aa9ZGGSQb2hZDQZmcVkMqab9BIaEH5rCOBGAyUu3KRzyENMlPSbSfk2VXPSW4tXYQNv7z+KoZP4tazY/UVBBMYzAfXBhmCjZLJLNn2IKUFK0Lv0CSaXqa8+zfSE9JRUol9oKpOW6pea1kmhMgvAEBaJcawnJhq6G/uywTjBkGXMM0wzzjcsNC5ha3ETewFtFN9CYijCUWxjZKi4U6wXW8xKWa/3RWmjKG3g59oM+GNrEo9uo1eA1os0+wg5pyhC2bsD3vCkGQ16NOksKCO3yDKmUyptR9m+yWjbZF4SIHMdWshboUHcFB4uW4baTI5AOUrzXIMK39LhusFqOmAdqJ5/NwO/pu3B+z1O1QLpi0IYB864pEO9bAc7s0mhGCa7wc3cUgIm6hL0CYYEo8vZj/WTRrARUgGfJ8/jC0JW6lbqn9M9p4/O1jaysJA47M2SmBrPXeqEYJbOpYpr7ysZ0vTxO6NWP/nph+wkg7Zl7avE+k2b1ktHQ9c9LQpYafXk9lX83Pk/rT0sPdx+vWLZshWUBflbafZ+TTbRwyhPkE7aBEtk5qEY7+EG5SzFRULY0pySTMHelDnxLZrkHu0ziIHivaX/gCxPCBijQWGKFK1XjB5jkfFFozEb1RhPOurkG+3XG9uvU+RuPceT1B2yhCJAL8rOTeCGo5QJRQeEGYPg1TBdfZDFVR592FEfV2dZExYIYdjNbDQERKPBNjyBrH76bHNKihbQ+jRcbmmj9fKBFuEsquE9s5Kjkp3J0cmu5Jjk2KGJniiP0xPtcXliPLEZURnOjOgMV0ZMRmxGYlHiiqgKZ0V0hasiZkXsukRf4s1EZ1fTrkZdDXKcOdE5rpyYImdRdJGrKKbMWRZd5iqL6XbnbnAv62+JS1ODTQJFz9SYO/OKUOntC3uXFD5fX1c39OjKvY3tPzHp5c05BzPz3p70t5tSan7J5OKPD/QY076kJt/77o63jllLV/fuXZOY2KZmqLP9l/AK2SochnoioZytlIPKzStN9Ra5PoyMFKG3mmGkbXiE0nY5pStCipbryvfXkz0BwZFKZFnkukhfJCdlte2qU+H+di0yduxXeGXstow3P/jgzYxtYx/and0uztM60j26Q07bm5R0qanpUlJSTXw8G8KCmJUNiiMPklZyKWllg0jK5uJpNhvLDSu5/VXG6wPZkW711rrANY5Iu2SwG2C0ZA0e7tCS+QYtY1WV7NhsWzrieI+hUUVRvqiPom5G8aEwlA2VhtqHRvKe+j6GPsaepkIoZIVSob0w0pg9WwUSo6ULGgYtrtP+q9fA6eXStv2BTYemnZg85aPpokWcYD3avmD6Omn3yi31QdLjk94+0bfvvrt6sgHMxELY/eKzhs0H9m1XLU2BXN6qs9GsPOrpYbBwHddbdDqOFpnLkpajG7iEeBQ2G3WcyToDM8CIADUWnG3Q8rXBzZe7wqB8rSsGqGUKAkqgGgR+I0smQ6iUKPXgSYYJUr401VAsLeBLpVX8WcMGqZpvNuyUrEZu1EkBaNJ3x0S5O0/SJek9gQWYE7gKV8ir+FpdpX4LbtbX4Mv8oP59/Xn9LbyJt+SbckT2bAqIqRaWatTC4eF6yf1t+z5p+s32E/U6W9sT7FJ7S/teKa79M8L7LoFeSHgRYg/BZklFc/s9zWNWuIdn8BxexG9yHVN7jXu3jl7K/6MZOv0/j/yvvruNone3eivUB9ap727W4EfQah/+i3c3T9zQ8BIo0ZXqSw2lxlJTaUBJYKm5NKg0uFQptZRYfeE3wy0/z65+9opXvHHva5s27N274Sazius3vxM3mAUvXDl16srVkye+3ipOimZxjZw9kHxqYwNIw8NigryLNFQozg/xREbUQ5Ctnhvqgtawt/BolMUa8ECYDAZphBbNUzp0pTy1QU1Zkz3GHGeZ0+f83CmzbDeFsVS7luXRqyctcdaVQJCmrLi+ftAbJafB7z9d8oY04OX1619W6ZX2fTpTTa5XHBU/0nnUy75tvHKlkajj25uU9fw9YWWG3wQP/h6iDdrHq98/HXS5637rfNuYoCyj+kXXcPtbF7XTzxRRAEHi1vnWR4Ky/uHbWITcpH1TAqmGaDXRJcil+yc8DCqILhJVE20lyiXaTlRJtJpoCV8Mp3RX4BSvhFPyJX+rfAVK5HyYLTcT7YDD0kB4V/4SZnObulZuH6H0OjIDdrJYVs3+Jo2T3sIEXIdX5Tz5Bk/kb+kMujm6byjDyNK/0oEBIjATkqCAsg961YXnVYSyXQqlu/r9SQ+T1G8OspGEk7XveGqZQShxHWUJDGxEZxnvqJfvKHPoxsZ2lnVgY/lwPxRCESyEOfTiNJVGn0uZWneYAj3ongLJdKZSaTJJuCCdZOZCMdEcyAMvzISeVPsgzCL53lS6jxDPoPu4230Va1we3fOozXy65pKk6X8war/bo6pfROfTWOpXplkkrerhpTb/uxGHUWkatZsA80hiCsl6td7ytBZeDZGLepmlfu8lmcnU7xMk56L2hTS6V3v2y37Ga70Uk0YUjWE61aqjFpNsodZTCo2dCmk/a9XVpvMfBf/T2rfsfzwitHmh/h+hUP4cQruK+l9FGL0hhNP+4qAsIZl67k99j6AX37HwMOEeD1AnlXn8PwlsteF/uPHHFLxVjT8E4fcCWwT+zY1/DcLvqvGmG288cx+/IfB6NV6rxuZW/LYVvxH49SC8mo5XBH6Vgl9eHs+/rMbLJHh5PF76og+/1Ipf9MGLAj8XeCEF/2LDz6rxU4GfWPHPi/HjI/gngedJ/PxiPHf2AX5uMZ59AM/8IZKfEfiHSPy9wI8E/rvAfxPYVI2nG538tMBGJ36YgqcEfrDCwj9w4Puh2CDwuMD3BL4r8JjAdwS+LfAtgUcFHhF42IL15W5eL7Du0BFeJ/DQwWx+6AgeKpMP/s7ND2Z7/HjQI//OjQcEvlmN+wW+IbBW4G8F7svF14Nw72tuvjcXX6ux8tfcWGPFV0npV1vxFYEvC9wjcLcVdwncuSOI70zBHUH4Ui76SMRXjS8K3L4tkG8XuC0Qt74Qzrfm4gtbFP5COG5R8HkTPidwc7WZbxZYbcZN1GhTNW7cEMQ3dscNQbi+FavWHeFVAtdVZvN1R3BdmVz5rJtXZmOlR37WjWsFrlndm68RuLo3PkMwn7kPV60M4KtsuDIAK6iiIhfLyVLlblxhweUCly218GUCl1pwicAygaUCPf6nFy/mTwtcvBifysWSTDsvceMigQsFPhmECwJxvgnnCZzbisWtOKcVZ7dikcBCgbMEzojB6QKnWdL5tPH4hMCCxTiVmHyBeQJzBU4ROFmgdxDmtOLjgZgt8DGBkwRmTTTxrFacaMJfhYbzX6XgBIGP0siPpmOmHcczhY/vhuNs+MioEP6IwIwAfFjg2IcUPlbgQwqOETianowWOOpBhY8KwQejzPxBBUea8QGBI6pxeDUOE3i/1Ivf34rpR/C+0egROFTgkHutfIgN7x0czO+14uB7zHywxx+M95hxkMCBAgf0t/EBrdi/n8L727BfWgDvp2BaAPZ1YqoZU+4O4CkC7w7A5D4BPNmMfQKwdy8j761gLyP2TMGku9w8KRfv6mHld7mxhxW7J7p59/sw0Y0J7gCeEIzuAIwXGCcwNhhjCGeMFV25GN2KToLgzMUoMzrIgg6Bka0YkY7hxIQL7JaLYWSpMIGh1Cg0HO0CbQJDBFpJwCrQQlgt6agsxuBcDBJoDgzlZoGBJB0YigECTQoaBRpIzCBQb0NdLsr0UKYZYEeqRUEvqAqXeiFTEASyOpa7Yi1L+v9wwP+1Av/tEfWf1zRPngplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjU4OTcKZW5kb2JqCjE2IDAgb2JqCjw8IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlIC9MZW5ndGggNjcgPj4Kc3RyZWFtCnicY2CgEDDjlGFhYAWSbAzsDBwMnAxcDNwMPEA+LwMfhkp+rPoF4CxBIBbCaY8wEIsAsSiSmBiDOJiWYJAEAEu2AUMKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvRmlsdGVyIC9GbGF0ZURlY29kZSAvTGVuZ3RoIDMyNyA+PgpzdHJlYW0KeJxdkktrhDAUhff+irucLgZHHactiFCmGxd9UNuVzEKTqwg1hugs/PdNchwHKujHOffhJTfhuXgtVD9T+GlGUfJMba+k4Wm8GsHUcNerIIpJ9mJelf+KodZBaIvLZZp5KFQ7BllG4ZcNTrNZaPcix4YfAiIKP4xk06uOdj/nElZ51fqXB1YzHYI8J8mtbfdW6/d6YAp98b6QNt7Py96W3TO+F80Uex1hJDFKnnQt2NSq4yA72CenrLVPHrCS/+JRgrKm3fJjlw9U4MXbEjav9iZ9NEFRckL0JiMgBhLgCKS3CjR4gnxeG2zSR4/o4FCBsDGEQwV6O0V2umana3aKHztUoLdPGNKhAmFjWocKhI2xHSoQdgNbrPYmEW29fFyP9C4vbiW3s3fbcVdpW724GmO37u+bX7dbdK94u5J61K7KvX/EY70iCmVuZHN0cmVhbQplbmRvYmoKMTQgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucwovQ0lEU3lzdGVtSW5mbyA8PCAvT3JkZXJpbmcgKElkZW50aXR5KSAvUmVnaXN0cnkgKEFkb2JlKSAvU3VwcGxlbWVudCAwID4+Ci9DSURUb0dJRE1hcCAxNiAwIFIgL0ZvbnREZXNjcmlwdG9yIDEzIDAgUiAvU3VidHlwZSAvQ0lERm9udFR5cGUyCi9UeXBlIC9Gb250IC9XIDE4IDAgUiA+PgplbmRvYmoKMTUgMCBvYmoKPDwgL0Jhc2VGb250IC9CTVFRRFYrRGVqYVZ1U2FucyAvRGVzY2VuZGFudEZvbnRzIFsgMTQgMCBSIF0KL0VuY29kaW5nIC9JZGVudGl0eS1IIC9TdWJ0eXBlIC9UeXBlMCAvVG9Vbmljb2RlIDE5IDAgUiAvVHlwZSAvRm9udCA+PgplbmRvYmoKMTMgMCBvYmoKPDwgL0FzY2VudCA5MjkgL0NhcEhlaWdodCAwIC9EZXNjZW50IC0yMzYgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC0xMDIxIC00NjMgMTc5NCAxMjMzIF0gL0ZvbnRGaWxlMiAxNyAwIFIKL0ZvbnROYW1lIC9CTVFRRFYrRGVqYVZ1U2FucyAvSXRhbGljQW5nbGUgMCAvTWF4V2lkdGggODYzIC9TdGVtViAwCi9UeXBlIC9Gb250RGVzY3JpcHRvciAvWEhlaWdodCAwID4+CmVuZG9iagoxOCAwIG9iagpbIDMyIFsgMzE4IF0gNDUgWyAzNjEgMzE4IF0gNDggWyA2MzYgNjM2IDYzNiA2MzYgNjM2IDYzNiA2MzYgXSA1NgpbIDYzNiA2MzYgXSA2NiBbIDY4NiBdIDc3IFsgODYzIF0gODIgWyA2OTUgXSA4NCBbIDYxMSBdIDk3IFsgNjEzIF0gOTkKWyA1NTAgXSAxMDEgWyA2MTUgXSAxMDcgWyA1NzkgMjc4IF0gMTExIFsgNjEyIDYzNSBdIF0KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL0NBIDAgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTIgPDwgL0NBIDEgL1R5cGUgL0V4dEdTdGF0ZSAvY2EgMSA+PgovQTMgPDwgL0NBIDAuOCAvVHlwZSAvRXh0R1N0YXRlIC9jYSAwLjggPj4gPj4KZW5kb2JqCjUgMCBvYmoKPDwgPj4KZW5kb2JqCjYgMCBvYmoKPDwgPj4KZW5kb2JqCjcgMCBvYmoKPDwgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL0NvdW50IDEgL0tpZHMgWyAxMSAwIFIgXSAvVHlwZSAvUGFnZXMgPj4KZW5kb2JqCjIxIDAgb2JqCjw8IC9DcmVhdGlvbkRhdGUgKEQ6MjAyMzAzMDYwNjQ1MDBaKQovQ3JlYXRvciAoTWF0cGxvdGxpYiB2My41LjMsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My41LjMpID4+CmVuZG9iagp4cmVmCjAgMjIKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMDg4NDUgMDAwMDAgbiAKMDAwMDAwODYwOCAwMDAwMCBuIAowMDAwMDA4NjQwIDAwMDAwIG4gCjAwMDAwMDg3ODIgMDAwMDAgbiAKMDAwMDAwODgwMyAwMDAwMCBuIAowMDAwMDA4ODI0IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MCAwMDAwMCBuIAowMDAwMDAxMjUxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMTIzMSAwMDAwMCBuIAowMDAwMDA4MTc4IDAwMDAwIG4gCjAwMDAwMDc4MTggMDAwMDAgbiAKMDAwMDAwODAzMSAwMDAwMCBuIAowMDAwMDA3Mjc5IDAwMDAwIG4gCjAwMDAwMDEyNzEgMDAwMDAgbiAKMDAwMDAwODQwMiAwMDAwMCBuIAowMDAwMDA3NDE4IDAwMDAwIG4gCjAwMDAwMDcyNTggMDAwMDAgbiAKMDAwMDAwODkwNSAwMDAwMCBuIAp0cmFpbGVyCjw8IC9JbmZvIDIxIDAgUiAvUm9vdCAxIDAgUiAvU2l6ZSAyMiA+PgpzdGFydHhyZWYKOTA1NgolJUVPRgo=\n",
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
"![\"DPR](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_dpr.png?raw=1\")
"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 337,
"referenced_widgets": [
"054fed8fdf48494db227ec97d40e7d50",
"0da52647fc5548b2b0f8f590bbd426fd",
"2214892e47cf43d5b187fa82b5ae1242",
"39f1b72c37444491aaf32fe24cd47487",
"f681cee7a15b450cbc1e7d7c600333ff",
"a748a6c9a63f42f8b05ea1579ea8e7ef",
"3153dba1224a4ed9ae9ddf358863e5e4",
"79459a18de4a47adbdfcf74d4f030167",
"6aa03d64c26b42ffb45412864401161c",
"4f855da5e66b4787a626805cef2422b2",
"09557a3024794fac85f3acae1f71a83d",
"f5341f6caba04e9bb7508a2e0fc317fc",
"a4eb739c64bf4508be68e0e881222a8b",
"89ac08e825494a43a7d6c071585127fa",
"0670fa053dc445e3831b771aae0f9aba",
"45f72e1ba86148d795227af866ae63c6",
"50d62ad707ce497eac491e36a7b8ba62",
"7a8a540d5b7d48e6943c4763e448ded5",
"0d61042d3303482c9c182714e8ba0787",
"32c66cf82d7145d1912fc007bfd85b2c",
"5e546217bc47462088c4f0edea85f8cf",
"38166990fa8145fbaccaf1449bcda745",
"8de3b97a338f4d2589f7eca645c042d8",
"7b19861cf8974f629d5b76f33eccafc9",
"10851a43255b47fc8a320c29afc750eb",
"07896cb770304c19b96f3328cd0e6a05",
"6401d537b91846c1b5910d5d4976b891",
"7bfc49f8f1d54b87b74f857445394b17",
"1a3ad624f8f84fae85b400746140561e",
"73a598f8b6e54ad58edb2cb35b84a06d",
"7d7d2ae292cb482792bea5877d2827e3",
"3f8a5983ff9146fc92d54b5f91d6dec1",
"cba72b421a434c38ba4ee8216c71726f",
"5f228abb79574183b2a6429ce137568b",
"579b7086ecf04fd6a64f4926cfa27ede",
"15c7e84cbe5a4a93956ed24795e20b98",
"0f98d03776ff419ebde75eacd5a66e9a",
"880e98851cd040a594cb186830c74ac5",
"2762e545eac8496eb2ef3faaf9341529",
"10f1d896b1ec4c55ac0bb3f1ab18a3ab",
"e92ae5560d804aef90d41bfc8f563e50",
"ac8ff19ec10d4ad08556f80ec009d0d9",
"43924d5dc07a4f09a10203263871929c",
"6f7f82ebb8024360a6e6ee613f3083fc",
"84c0d8ed2346467d933c690396d9b8ed",
"8903ff45785942379e05bbe826631ef0",
"9e47fefdef41443fa897add035a46835",
"4dd13fd377bd45c6bbe090c142134b60",
"04f6caee9e684ed3bdb0032c9e0cff02",
"52046842098e46c2ad0ad26aae21df5d",
"74a67cf72d3c47ccb78849399806bb81",
"94b2e2c478fd480eae6369ad602ac734",
"ae4a61e58c0346449e61515ae85d42e1",
"49ea6bea756a46d4a8fc280479d69478",
"b43e12739ec6452b89704fc7789462ed",
"5a6b82e900144a2bbc60f7d173acd1fe",
"c4764c0f3a0b4c5090fd5ca6057b792b",
"d13d3df595874e5e8015d782baf2ba02",
"c5af7e58db404bfc8c6b4a47a1957f1f",
"422ef12185db47d9879f6fd716b08b20",
"6551aaa6d7ff45dfb56d3ec0c588c442",
"2cb2fb24787f4f9fa8383df806cb3cfd",
"0668943d9d464fe5bde5ce3a332cd1a2",
"e1b003b8e0e94f22b9258e503c1b093d",
"7da41ceef70e42caabf414ec54ad58e1",
"3e07697f1bbe414ab02144779ce060b2",
"efeafc8d24e2454d86f2cc840d5b8c7d",
"c3584c6d901849c4abcf664748fd0ace",
"056dd79a76fa4e6aa08ccda715b4988e",
"824bd3f969e744b1b032bcebe86d12de",
"d454994e61bd47169dfde9a0d9645dd9",
"f39ae404fba94241aa40b11ffba5f681",
"a354f24998874e4887fff4c3aa1c43f7",
"52184bae537946faa068d0ce469a8b2b",
"921be580aefd4c90ab6942f94e956a8f",
"ced15d6b54444cba878af934094f4eb4",
"8e25dfd6e28e46f5835caecb932db111",
"1e1dc199299048b4ba8241261caf39c9",
"945833f08a46463d89e15419a8ec28ba",
"a58e70a4776842ebb1000f8d50eb2080",
"288efc2a53e8484a9b1f9ca7a1ef3ca7",
"54f05d93434947fe9e601b992cbe3619",
"46308fb8060a477b9259b450ee02b9fc",
"1a6ad8fce08a42e08aa267d9a935c93f",
"45a98350b39046fd94fe6650e46ee21f",
"ef2814aabfc04157b526e58295ef1f09",
"9a46c093570b4e33a32287a1da117cca",
"3031cd0717d8455aba0e35d01a27c5fe",
"b0cc841385c9468787653f229a07bbe2",
"262be61e3850460da51188b521dfeadb",
"26b60259fbc34e79a05080857deea519",
"507d323d148242baa6f2561d5ff4e002",
"13b1807e63a2495d93bc9808c9da3cc5",
"b270a828f8714abf8c125321b14ea07f",
"8169189b38e74b218c62782eafbf9ebb",
"606f82d23ab64a209c621c04e702b4db",
"3df00e02523e4d478a6a1e3761b05130",
"fdf00a7d26ba4131ae96da90986b7779",
"73655c00d83846e8a8bfab0fde5d822d",
"4369bba69270488ca1a9ed6e1c06f693",
"59c5854da5bd4971ae72c2601a371830",
"3abe8cf41fdf47078815b03a7df8ea90",
"4bfcdccdc5544405b4788871b3fdeb3a",
"a836b9ff68dd4d5e8cd25f2622e067a0",
"1eb63083884f4f8dbd78200f50be8204",
"f08e7f6394754d40bd70ffcae226667e",
"1b1f9306777b4fedb6306b1c89273615",
"5a8346dc00df4777bf7566db52a3a50f",
"9eee1fdf30954be7a7b6da68846a96f1",
"46ca21a58ea0489a9c5c678825477b5f"
]
},
"id": "jyqMIRf7z42D",
"outputId": "3992b072-5d43-436c-9065-83f564c15eb8"
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "054fed8fdf48494db227ec97d40e7d50",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Downloading: 0%| | 0.00/232k [00:00\n",
"\n",
"\n"
],
"text/plain": [
"![\"SQuAD](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_squad-schema.png?raw=1\")
"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fhfruh1kz42F"
},
"outputs": [],
"source": [
"def create_paragraphs(df):\n",
" paragraphs = []\n",
" id2context = dict(zip(df[\"review_id\"], df[\"context\"]))\n",
" for review_id, review in id2context.items():\n",
" qas = []\n",
" # Filter for all question-answer pairs about a specific context\n",
" review_df = df.query(f\"review_id == '{review_id}'\")\n",
" id2question = dict(zip(review_df[\"id\"], review_df[\"question\"]))\n",
" # Build up the qas array\n",
" for qid, question in id2question.items():\n",
" # 하나의 질문 ID에 대해 필터링합니다\n",
" question_df = df.query(f\"id == '{qid}'\").to_dict(orient=\"list\")\n",
" ans_start_idxs = question_df[\"answers.answer_start\"][0].tolist()\n",
" ans_text = question_df[\"answers.text\"][0].tolist()\n",
" # 답변 가능한 질문을 추가합니다\n",
" if len(ans_start_idxs):\n",
" answers = [\n",
" {\"text\": text, \"answer_start\": answer_start}\n",
" for text, answer_start in zip(ans_text, ans_start_idxs)]\n",
" is_impossible = False\n",
" else:\n",
" answers = []\n",
" is_impossible = True\n",
" # 질문-답 쌍을 qas에 추가합니다\n",
" qas.append({\"question\": question, \"id\": qid,\n",
" \"is_impossible\": is_impossible, \"answers\": answers})\n",
" # 문맥과 질문-답 쌍을 paragraphs에 추가합니다\n",
" paragraphs.append({\"qas\": qas, \"context\": review})\n",
" return paragraphs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "OIRacuduz42F",
"outputId": "4dbbe468-b6d0-4bea-886a-5871e8303abc"
},
"outputs": [
{
"data": {
"text/plain": [
"[{'qas': [{'question': 'How is the bass?',\n",
" 'id': '2543d296da9766d8d17d040ecc781699',\n",
" 'is_impossible': True,\n",
" 'answers': []}],\n",
" 'context': 'I have had Koss headphones in the past, Pro 4AA and QZ-99. The Koss Portapro is portable AND has great bass response. The work great with my Android phone and can be \"rolled up\" to be carried in my motorcycle jacket or computer bag without getting crunched. They are very light and do not feel heavy or bear down on your ears even after listening to music with them on all day. The sound is night and day better than any ear-bud could be and are almost as good as the Pro 4AA. They are \"open air\" headphones so you cannot match the bass to the sealed types, but it comes close. For $32, you cannot go wrong.'},\n",
" {'qas': [{'question': 'Is this music song have a goo bass?',\n",
" 'id': 'd476830bf9282e2b9033e2bb44bbb995',\n",
" 'is_impossible': False,\n",
" 'answers': [{'text': 'Bass is weak as expected', 'answer_start': 1302},\n",
" {'text': 'Bass is weak as expected, even with EQ adjusted up',\n",
" 'answer_start': 1302}]}],\n",
" 'context': 'To anyone who hasn\\'t tried all the various types of headphones, it is important to remember exactly what these are: cheap portable on-ear headphones. They give a totally different sound then in-ears or closed design phones, but for what they are I would say they\\'re good. I currently own six pairs of phones, from stock apple earbuds to Sennheiser HD 518s. Gave my Portapros a run on both my computer\\'s sound card and mp3 player, using 256 kbps mp3s or better. The clarity is good and they\\'re very lightweight. The folding design is simple but effective. The look is certainly retro and unique, although I didn\\'t find it as comfortable as many have claimed. Earpads are *very* thin and made my ears sore after 30 minutes of listening, although this can be remedied to a point by adjusting the \"comfort zone\" feature (tightening the temple pads while loosening the ear pads). The cord seems to be an average thickness, but I wouldn\\'t get too rough with these. The steel headband adjusts smoothly and easily, just watch out that the slider doesn\\'t catch your hair. Despite the sore ears, the phones are very lightweight overall.Back to the sound: as you would expect, it\\'s good for a portable phone, but hardly earth shattering. At flat EQ the clarity is good, although the highs can sometimes be harsh. Bass is weak as expected, even with EQ adjusted up. To be fair, a portable on-ear would have a tough time comparing to the bass of an in-ear with a good seal or a pair with larger drivers. No sound isolation offered if you\\'re into that sort of thing. Cool 80s phones, though I\\'ve certainly owned better portable on-ears (Sony makes excellent phones in this category). Soundstage is very narrow and lacks body. A good value if you can get them for under thirty, otherwise I\\'d rather invest in a nicer pair of phones. If we\\'re talking about value, they\\'re a good buy compared to new stock apple buds. If you\\'re trying to compare the sound quality of this product to serious headphones, there\\'s really no comparison at all.Update: After 100 hours of burn-in time the sound has not been affected in any appreciable way. Highs are still harsh, and bass is still underwhelming. I sometimes use these as a convenience but they have been largely replaced in my collection.'},\n",
" {'qas': [{'question': 'How is the bass?',\n",
" 'id': '455575557886d6dfeea5aa19577e5de4',\n",
" 'is_impossible': False,\n",
" 'answers': [{'text': 'The only fault in the sound is the bass',\n",
" 'answer_start': 650}]}],\n",
" 'context': \"I have had many sub-$100 headphones from $5 Panasonic to $100 Sony, with Sennheiser HD 433, 202, PX100 II (I really wanted to like these PX100-II, they were so very well designed), and even a Grado SR60 for awhile. And what it basically comes down to is value. I have never heard sound as good as these headphones in the $35 range, easily the best under $75. I can't believe they're over 25 years old.It's hard to describe how much detail these headphones bring out without making it too harsh or dull. I listen to every type of music from classical to hip hop to electronic to country, and these headphones are suitable for all types of music. The only fault in the sound is the bass. It's just a *slight* bit boomy, but you get to like it after a while to be honest.The design is from the 80s as you all have probably figured out. It could use a update but it seems like Koss has tried to perfect this formula and failed in the past. I don't really care about the looks or the way it folds up or the fact that my hair gets caught up in it (I have very short hair, even for a male).But despite it's design flaws, it's the most comfortable headphones I have ever worn, and the best part is that it's also the best sounding pair of headphones I have ever heard under $75.If you can get over the design flaws or if sound is the most important feature of headphones for you, there is nothing even close to this at this price range.This one is an absolute GEM. I loved these so much I ordered two of the 25th Anniversary ones for a bit more.Update: I read some reviews about the PX100-II being much improved and better sounding than the PortaPro. Since the PX100-II is relatively new, I thought I'd give it another listen. This time I noticed something different. The sound is warm, mellow, and neutral, but it loses a lot of detail at the expense of these attributes. I still prefer higher-detail Portapro, but some may prefer the more mellow sound of the PX100-II.Oh by the way the Portapro comes in the straight plug now, not the angled plug anymore. It's supposed to be for better compatibility with the iPods and iPhones out there.\"}]"
]
},
"execution_count": 58,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"product = dfs[\"train\"].query(\"title == 'B00001P4ZH'\")\n",
"create_paragraphs(product)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sbLyB3WNz42F"
},
"source": [
"```python\n",
"[{'qas': [{'question': 'How is the bass?',\n",
" 'id': '2543d296da9766d8d17d040ecc781699',\n",
" 'is_impossible': True,\n",
" 'answers': []}],\n",
" 'context': 'I have had Koss headphones ...',\n",
" 'id': 'd476830bf9282e2b9033e2bb44bbb995',\n",
" 'is_impossible': False,\n",
" 'answers': [{'text': 'Bass is weak as expected', 'answer_start': 1302},\n",
" {'text': 'Bass is weak as expected, even with EQ adjusted up',\n",
" 'answer_start': 1302}]}],\n",
" 'context': 'To anyone who hasn\\'t tried all ...'},\n",
" {'qas': [{'question': 'How is the bass?',\n",
" 'id': '455575557886d6dfeea5aa19577e5de4',\n",
" 'is_impossible': False,\n",
" 'answers': [{'text': 'The only fault in the sound is the bass',\n",
" 'answer_start': 650}]}],\n",
" 'context': \"I have had many sub-$100 headphones ...\"}]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fHRaCmFrz42G"
},
"outputs": [],
"source": [
"import json\n",
"\n",
"def convert_to_squad(dfs):\n",
" for split, df in dfs.items():\n",
" subjqa_data = {}\n",
" # 각 제품 ID에 대해 `paragraphs`를 만듭니다\n",
" groups = (df.groupby(\"title\").apply(create_paragraphs)\n",
" .to_frame(name=\"paragraphs\").reset_index())\n",
" subjqa_data[\"data\"] = groups.to_dict(orient=\"records\")\n",
" # 결과를 디스크에 저장합니다\n",
" with open(f\"electronics-{split}.json\", \"w+\", encoding=\"utf-8\") as f:\n",
" json.dump(subjqa_data, f)\n",
"\n",
"convert_to_squad(dfs)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "flhLYefGz42G",
"outputId": "0624715e-14a1-45b5-d73c-a02f0411dd42"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\r",
"Preprocessing Dataset electronics-train.json: 0%| | 0/1265 [00:00\n",
"\n",
"\n"
],
"text/plain": [
"![\"RAG](\"https://github.com/rickiepark/nlp-with-transformers/blob/main/images/chapter07_rag-architecture.png?raw=1\")
"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 361,
"referenced_widgets": [
"24201c4e037c4521b0eddbc5acf14691",
"6aa552d8122b4f1d900b2b0e1bbce2b3",
"5368a0d8431a4f78aa8177e435a02044",
"2cb6cd55021d4df786e9be9cf0fd1b6d",
"33d160d0403e4752b8ccbc21771a6982",
"91ed881e47304d19a43ca418f365d8d6",
"597ba649de0347bdbd6bb206f5b9ffae",
"324997655bb048908642cf935d5051b0",
"35d6d5b4796146f29d537e6a66fa959e",
"35fbba550fbe4b32afc9d37616b99de0",
"1dda099c6e214b5aa17086d6334524c2",
"97e1653fecbf4fe6b09cefc998dda8f6",
"2303540a318a41cab13728254bdd93c8",
"b2b7957de17c4a98bfaae26addf28860",
"a96f7ee2514b43279c12713160afaa78",
"24ea75b4190449c7ba3eaebc1e3ed920",
"90458bdcaafb4c9a9a58ce9f68596ad5",
"0b4490a071da4343927f1de85077fbf9",
"5f6a69a0c7fb40639baff3d21330c5bc",
"410484c5dc674bae851a59855b1e64ca",
"6c42b3d197cd43b4819d7b54519fe828",
"7df205b130f9448da4ad935d01487f2a",
"3b1ae7324a1449308a91efac7920a217",
"f3615131d88240afaa1ac0720aede7c4",
"d0967753213f42abac4741b7bb2fd08e",
"0caa719ea1084f178ae38f8a7df61881",
"cdd53d905acb4405acd0c338d36376ba",
"ab68f3fe206f4ca999bfc699f3fc26dd",
"699d545cd351410c94d7a2b9919073f9",
"3c531571972c453184084c188351196e",
"a14f56f3525e4244988aa23522c29dc4",
"ab41baffed8549f3a2d10780522ec6d2",
"fb3fbb598e2b4a2b946f8e38537f9ecd",
"da0771fd53424bfea4813287d7596ba2",
"9d608880b7074d118f06ba4580cc830e",
"e6d8d1c99b124815a677287f3a23d855",
"a176bb76c2224caa9ed5b6eae65f1564",
"a07930d28d774f0fbb6b49baa18e4197",
"f9b9185d995f42ecab490619ca56f869",
"8669ecdf967b49f3b1228ac2796fb7a9",
"50964575a89f41968d1e12256c5042e1",
"bf84ed5045984fe98eb4ace77e0b4d67",
"ef436542e0a744ac983d89656390c1f5",
"ada018dd49974dc58324a31802455be4",
"4dde3fbfcb2944c3ab12694c87a4a7ee",
"23f40f453b794fd6b3bdde3e2b4a590e",
"9dc2a12ad0c8492dadefcb4df8c11727",
"965384baefcd41a6a8f84bc1595c586a",
"c234eec2de7c4398bbba485dfd546b8d",
"1424e0fbd853472c9ba2b10fb1ee7053",
"d7a99d0c739f4ae797281c7f6f1692d1",
"9e69fe0c6c684ae6951c6e7e25167b88",
"68227be14ac44ee9926ae5e38325610a",
"e8ecf9f62d3041c4a38927c6882620cb",
"579a7f13bf63462bab5d37fc33915448",
"03e428e2e73049bb905969eff9580a44",
"b60acd0208864b22a27fbef1de7df37f",
"b14e5833789b48cfa0184bb1f96cb42b",
"b4aab01faeca457ca5753de0cbf7adde",
"35de3d53e98f4542ba00af58a2755f8c",
"b5e861339e54462c933b21894cc2d8a6",
"7bce87515ea6443298a392574a71f932",
"dac07ee151724d6e9a35ba4ce914ee05",
"d8f84a5f1c2f4994961cd018314e54e0",
"ecfe3ef8abcd43e2992abfa188a7c30e",
"e9383fa4554749c4a7c20acf2e47f288",
"7279382f89ab43e5b8b06ebf11e1d3b3",
"8107637926474da4922e13d33698a1c1",
"2843bab33dcc4b9eae01a3661f884489",
"8f9a4093735d4b468c7842f828838791",
"02893b7c50d4483ba01adeb66e3415cb",
"4ea98f5f71ea45d990b7e5eaf0313ca9",
"2d6af5d950df40199a89a7acf6052ad5",
"5586941950c24573b73466007cf30510",
"234fa67a0c554eae837ff0805f6a6238",
"32eb73fcab60469bbe7614bd62df8d4e",
"84c13f0bf353402b8e0f044216692a52",
"c774b980469840708af26b970509a643",
"44d8ab80464a4d2c846e5815b0e109e9",
"b07e0284bd454d679cc832c75f406266",
"a4f847c2ed714642a9e5fd860ae1b9f3",
"ecec445e15a440acadd23469083cbf81",
"e06482a4537a409ab529ffc64ad59535",
"4571c4ac68d94071a758bed8c0bbaddf",
"e246ae2b88ec4806ad5dbe82094a1d1b",
"3a49d109ece24d7687f77df7a1ddae18",
"e367cef0cc944c9ab138f81e5128b215",
"fef4af0a4bc74982976c6cbf82d2e277",
"d7e566bc809f456c8bec5eb1db2b34d6",
"54bb7257f2bf46d5855aef925fe59fb3",
"f741f6f629514e6e9b8a63be5f0cf2b7",
"7ccd1d52c0524d4499965ba69e6a9530",
"834c458c2c5347f29d0b24eecf319861",
"e3716150c4064df9a3d288fb509635d8",
"e92e4c674bee4db08053d584bdeef3e8",
"6222d158304540ba85d43952f5b00022",
"3d82104ae4d1473792669d781c812bc6",
"d2e1799132f2428a849647aa4a46f83a",
"0e5cf1e295fb47d2aed2ab3a4f8c9f31"
]
},
"id": "VGryx5itz42I",
"outputId": "ca7492e0-feea-4897-abd4-52fb00195518"
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "24201c4e037c4521b0eddbc5acf14691",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Downloading: 0%| | 0.00/4.60k [00:00"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"machine_shape": "hm",
"provenance": []
},
"gpuClass": "standard",
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.11"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}