\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
012
0how are glacier caves formed ?A partly submerged glacier cave on Perito More...0
1how are glacier caves formed ?The ice facade is approximately 60 m high0
2how are glacier caves formed ?Ice formations in the Titlis glacier cave0
3how are glacier caves formed ?A glacier cave is a cave formed within the ice...1
4how are glacier caves formed ?Glacier caves are often called ice caves , but...0
\n", "
" ], "text/plain": [ " 0 \\\n", "0 how are glacier caves formed ? \n", "1 how are glacier caves formed ? \n", "2 how are glacier caves formed ? \n", "3 how are glacier caves formed ? \n", "4 how are glacier caves formed ? \n", "\n", " 1 2 \n", "0 A partly submerged glacier cave on Perito More... 0 \n", "1 The ice facade is approximately 60 m high 0 \n", "2 Ice formations in the Titlis glacier cave 0 \n", "3 A glacier cave is a cave formed within the ice... 1 \n", "4 Glacier caves are often called ice caves , but... 0 " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "raw_train = pd.read_csv(os.path.join('WikiQACorpus', 'WikiQA-train.txt'), sep='\\t', header=None)\n", "raw_train.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Clearly, it is not in the format `deepmatcher` wants its input data to be in - this file has no column names, no ID column, and its not a CSV file. Let's fix that:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
left_valueright_valuelabel
id
0how are glacier caves formed ?A partly submerged glacier cave on Perito More...0
1how are glacier caves formed ?The ice facade is approximately 60 m high0
2how are glacier caves formed ?Ice formations in the Titlis glacier cave0
3how are glacier caves formed ?A glacier cave is a cave formed within the ice...1
4how are glacier caves formed ?Glacier caves are often called ice caves , but...0
\n", "