{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": false }, "source": [ "##Cleaning the Highscores\n", "\n", "There is a lot of cleaning that needs to be done before the highscores can be read into a pandas dataframe, not all of which can be done programmatically. \n", "\n", "I started by taking a copy of the HighScores.vcd file (may appear as a .dat file), opening it with Notepad (PSPad will open it in the hex editor) and saving it as HighScores.txt. Manually I delete everything down as far as the first occurance of ===, this looks something like: Assembly-CSharp\u0005\u0001\u0013XRL.Core.Scoreboard\u0001\u0006Scores\u0003�\u0001System.Collections.Generic.List`1[[XRL.Core.ScoreEntry, Assembly-CSharp....\n", "\n", "There are a number of symbols which will prevent the file being read into ipython fully or will cause trouble when writing the data to file and these need to be deleted manually as well. These symbols are included in notes.txt, which should be opened in Notepad, and these can be removed by using Edit -> Replace. There is also a circle shape that needs to be removed, which can be hard to find. This is usually the symbol for the bits that make up an artifact, so it may be best to delete everything between a < >. If going through this code using your own highscores you may have to manually delete more lines or symbols.\n", "\n", "After all that the file can finally be read in!" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "=== Game summary for &WGoethe II&y ===\n", "Game ended Thursday, August 13, 2015 at 6:04:58 PM\n", "Goethe II died on the 20th of Uru Ux.\n", "from &yWahmahcalcalit&y's lase beam! (x2)\n", "Scored &C48753&y points\n", "Survived for 35235 turns.\n", "Visited 260 zones.\n", "Generated 1 storied items.\n", "Most advanced artifact in possession: HE Missile&y\n", "\t\n", "\n", "< \u0001\u0006 \u0005 � \u0006= �\u0002=== Game summary for &WKant XVIII&y ===\n", "Game ended Sunday, August 30, 2015 at 7:34:00 PM\n", "Kant XVIII died on the 27th of Tuum Ut.\n", "The &ychute crab&y hits &w(x1)&y for 2 damage with his &Ycrab claw ->&y7 &r\u0003&y1d2&y!&y [7]\n", "Scored &C40178&y points\n", "Survived for 37145 turns.\n", "Visited 222 zones.\n", "Generated 1 storied items.\n", "Most advanced artifact in possession: Fix-It spray foam x2&y\n", "\t\n", "\n", "= \u0001\u0007 \u0005 LP \u0006> �\u0003=== Game summary for &WO`Brien III&y ===\n", "Game ended Wednesday, September 02, 2015 at 3:50:10 AM\n", "O`Brien III died on the 6th of Tebet Ux.\n", "The &rbloody&y &MKumukokumu the Stylish, &Mlegendary &Mogre ape&y&y hits &M(x8)&y for 51 damage with his &Yape fist &c ->&y20 &r\u0003&y3d3&y!&y [10]\n", "Scored &C20556&y points\n", "Survived for 21114 turns.\n", "Visited 130 zones.\n", "Most advanced artifact in possession: force bracelet &b\u0004&y0 &K\t&y0 &y<&b\u0007&c\u0007&R\u0007&B\u0007&y> &y[&Kno cell&y]&y\n", "\t\n", "> \u0001\b \u0005 �B \u0006? �\u0003=== Game summary for &WKant XII&y ===\n", "Game ended Friday, August 28, 2015 at 11:14:48 PM\n", "Kant XII died on the 7th of Iyur Ut.\n", "&YPutus Templar warden&y hits &w(x1)&y for 3 damage with his &Bfolded carbide&y long sword ->&y9 &r\u0003&y2d5&y!&y [20]\n", "Scored &C17061&y points\n", "Survived for 17066 turns.\n", "Visited 118 zones.\n", "Most advanced artifact in possession: &Welectro&cbow &y<&g\u0007&c\u0007&G\u0007&G\u0007&y> ->&y10 &r\u0003&y1d6 &y[&Kno cell&y]&y\n", "\t\n", "? \u0001\t \u0005 �@ \u0006@ �\u0003=== Game summary for &WNietzsche III&y ===\n", "Game ended Wednesday, August 05, 2015 at 8:00:46 PM\n", "Nietzsche III died on the 19th of Tishru ii Ux.\n", "The &Yeyeless king crab&y hits &M(x6)&y for 20 damage with his &Ymassive king crab claw ->&y20 &r\u0003&y1d6&y!&y [11]\n", "Scored &C16607&y points\n", "Survived for 16124 turns.\n", "Visited 115 zones.\n", "Most advanced artifact in possession: &cu&gb&Ge&Wr&wn&co&Cs&Gt&gr&wu&Wm&y injector &y<&K\u0007&C\u0007&y>&y\n", "\t@ \u0001\n", " \u0005 �& \u0006A �\u0002\n", "\n", "=== Game summary for &WKant XI&y ===\n", "Game ended Thursday, August 27, 2015 at 11:14:09 PM\n", "Kant XI died on the 12th of Uru Ux.\n", "from the scalding steam!\n", "Scored &C9971&y points\n", "Survived for 11105 turns.\n", "Visited 60 zones.\n", "Most advanced artifact in possession: Fix-It spray foam&y\n", "\t\n", "A \u0001\u000b", " \u0005 �\u001e", " \u0006B �\u0002=== Game summary for &WKant VIII&y ===\n", "Game ended Thursday, August 27, 2015 at 8:01:17 PM\n", "Kant VIII died on the 23rd of Tishru i Ux.\n", "from the scalding steam!\n", "Scored &C7848&y points\n", "Survived for 7719 turns.\n", "Visited 35 zones.\n", "Most advanced artifact in possession: Fix-It spray foam&y\n", "\t\n", "\n", "B \u0001\f", " \u0005 \u0006-> \u0006C �\u0002=== Game summary for &WKant V&y ===\n", "Game ended Saturday, August 22, 2015 at 6:22:36 PM\n", "Kant V died on the 28th of Tishru i Ux.\n", "\n", "Scored &C6662&y points\n", "Survived for 8118 turns.\n", "Visited 47 zones.\n", "Most advanced artifact in possession: semi-automatic pistol x2 &y<&b\u0007&G\u0007&G\u0007&B\u0007&C\u0007&y> ->&y8 &r\u0003&y1d6 &y[&KEmpty&y]&y\n", "\tC \u0001\r", " \n", "\u0005 _\u0015 \u0006D �\u0002=== Game summary for &WO`Brien II&y ===\n", "Game ended Wednesday, September 02, 2015 at 12:53:32 AM\n", "O`Brien II died on the 23rd of Nivvun Ut.\n", "from &yDuhmahcaluhcal&y's lase beam! (x3)\n", "Scored &C5471&y points\n", "Survived for 6356 turns.\n", "Visited 42 zones.\n", "Most advanced artifact in possession: &cstun &cgas grenade mk I&y\n", "\tD \u0001\u000e \u0005 a\f", " \u0006E �\u0002=== Game summary for &WKant XVII&y ===\n", "Game ended Saturday, August 29, 2015 at 1:44:21 AM\n", "Kant XVII died on the 27th of Tishru ii Ux.\n", "\n", "Scored &C3169&y points\n", "Survived for 7239 turns.\n", "Visited 37 zones.\n", "Most advanced artifact in possession: compass bracelet &b\u0004&y0 &K\t&y0&y\n", "\tE \u0001\u000f \u0005 D\t \u0006F �\u0002=== Game summary for &WKant&y ===\n", "Game ended Friday, August 21, 2015 at 11:08:22 PM\n", "Kant died on the 12th of Tuum Ut.\n", "The giant amoeba&y hits &W(x2)&y for 4 damage with his &Ggiant pseudopod ->&y10 &r\u0003&y1d3&y!&y [15]\n", "Scored &C2372&y points\n", "Survived for 3013 turns.\n", "Visited 21 zones.\n", "\tF \u0001\u0010 \u0005 9\u0007 \u0006G �\u0002=== Game summary for &WKant XIII&y ===\n", "Game ended Saturday, August 29, 2015 at 12:05:24 AM\n", "Kant XIII died on the 5th of Tebet Ux.\n", "The girshling&y hits &w(x1)&y for 6 damage with his claw ->&y2 &r\u0003&y1d6&y!&y [12]\n", "Scored &C1849&y points\n", "Survived for 2853 turns.\n", "Visited 18 zones.\n", "Most advanced artifact in possession: pump shotgun ->&y8 &r\u0003&y1d2 &y[shotgun shell&y]&y\n", "\tG \u0001\u0011 \u0005 �\u0006 \u0006H �\u0001=== Game summary for &WKhrushchev VIII&y ===\n", "Game ended Sunday, August 02, 2015 at 6:19:47 PM\n", "Khrushchev VIII died on the 5th of Tuum Ut.\n", "from bleeding!\n", "Scored &C1760&y points\n", "Survived for 4198 turns.\n", "Visited 25 zones.\n", "\tH \u0001\u0012 \u0005 �\u0005 \u0006I �\u0002=== Game summary for &WNietzsche&y ===\n", "Game ended Tuesday, August 04, 2015 at 8:18:00 PM\n", "Nietzsche died on the 17th of Tuum Ut.\n", "The &rbloody&y horned chameleon&y hits &w(x1)&y for 4 damage with his Tusks ->&y4 &r\u0003&y2d3 &b\u0004&y0 &K\t&y0&y!&y [7]\n", "Scored &C1503&y points\n", "Survived for 3719 turns.\n", "Visited 27 zones.\n", "\tI \u0001\u0013 \u0005 q\u0005 \u0006J �\u0002=== Game summary for &WKant IV&y ===\n", "Game ended Saturday, August 22, 2015 at 12:00:38 AM\n", "Kant IV died on the 28th of Iyur Ut.\n", "The snapjaw scavenger&y hits &w(x1)&y for 6 damage with his &Ysteel&y battle axe ->&y3 &r\u0003&y1d6&y!&y [18]\n", "Scored &C1393&y points\n", "Survived for 3347 turns.\n", "Visited 16 zones.\n", "Most advanced artifact in possession: &Gacid &cgas grenade mk I&y\n", "\tJ \u0001\u0014 \u0005 p\u0004 \u0006K �\u0002=== Game summary for &WKhrushchev XI &y ===\n", "Game ended Monday, August 03, 2015 at 3:18:32 PM\n", "Khrushchev XI died on the 20th of Kisu Ux.\n", "\n", "Scored &C1136&y points\n", "Survived for 4121 turns.\n", "Visited 21 zones.\n", "Most advanced artifact in possession: &rbl&Ra&Wz&Ye&y injector&y\n", "\tK \u0001\u0015 \u0005 �\u0003 \u0006L �\u0002=== Game summary for &WLenin&y ===\n", "Game ended Saturday, August 01, 2015 at 4:08:30 PM\n", "Lenin died on the 13th of Iyur Ut.\n", "The equimax&y hits &r(x3)&y for 9 damage with his bite ->&y8 &r\u0003&y2d2&y!&y [17]\n", "Scored &C902&y points\n", "Survived for 1019 turns.\n", "Visited 12 zones.\n", "\tL \u0001\u0016 \u0005 �\u0002 \u0006M �\u0002=== Game summary for &WKhrushchev&y ===\n", "Game ended Sunday, August 02, 2015 at 3:52:52 PM\n", "Khrushchev died on the 10th of Tishru ii Ux.\n", "The &rbloody&y &MGroubuubu-wof-wofuz, the stalwart Snapjaw Tot-eater&y hits &W(x2)&y for 11 damage with his &bcarbide&y battle axe ->&y5 &r\u0003&y2d3&y!&y [22]\n", "Scored &C708&y points\n", "Survived for 2789 turns.\n", "Visited 21 zones.\n", "\tM \u0001\u0017 \u0005 \t\u0001 \u0006N �\u0002=== Game summary for &WKant X&y ===\n", "Game ended Thursday, August 27, 2015 at 9:17:34 PM\n", "Kant X died on the 8th of Kisu Ux.\n", "The cave spider&y hits &w(x1)&y for 2 damage with his fangs ->&y2 &r\u0003&y1d2&y!&y [19]\n", "Scored &C265&y points\n", "Survived for 2287 turns.\n", "Visited 16 zones.\n", "\tN \u0001\u0018 \u0005 � \u0006O �\u0003=== Game summary for &WO`Brien IV&y ===\n", "Game ended Wednesday, September 02, 2015 at 12:47:21 PM\n", "O`Brien IV died on the Ides of Uru Ux.\n", "The salthopper&y hits &r(x3)&y for 7 damage with his &Grending mandibles ->&y11 &r\u0003&y1d4&y!&y [15]\n", "Scored &C199&y points\n", "Survived for 1810 turns.\n", "Visited 14 zones.\n", "Most advanced artifact in possession: semi-automatic pistol ->&y8 &r\u0003&y1d6 &y[&KEmpty&y]&y\n", "\tO \u0001\u0019 \u0005 \r", " \u0006P �\u0001=== Game summary for &WKhrushchev II&y ===\n", "Game ended Sunday, August 02, 2015 at 4:13:32 PM\n", "Khrushchev II died on the 17th of Uru Ux.\n", "from bleeding!\n", "Scored &C13&y points\n", "Survived for 1640 turns.\n", "Visited 9 zones.\n", "\tP \u0001-> \u0005 ����\u0006Q �\u0002=== Game summary for &WStalin&y ===\n", "Game ended Saturday, August 01, 2015 at 3:28:05 PM\n", "Stalin died on the 27th of Uru Ux.\n", "The salthopper&y hits &r(x3)&y for 9 damage with his &Grending mandibles ->&y11 &r\u0003&y1d4&y!&y [16]\n", "Scored &C-71&y points\n", "Survived for 3412 turns.\n", "Visited 30 zones.\n", "\tQ \u0001\u001b \u0005 ����\u0006R �\u0002=== Game summary for &WKant IX&y ===\n", "Game ended Thursday, August 27, 2015 at 8:51:15 PM\n", "Kant IX died on the 2nd of Iyur Ut.\n", "The &gjilted lover&y hits &w(x1)&y for 2 damage with his thorns ->&y5 &r\u0003&y1d4&y!&y [16]\n", "Scored &C-80&y points\n", "Survived for 1807 turns.\n", "Visited 13 zones.\n", "Most advanced artifact in possession: &Gacid &cgas grenade mk I x2&y\n", "\tR \u0001\u001c", " \u0005 ����\u0006S �\u0001=== Game summary for &WKhrushchev VI&y ===\n", "Game ended Sunday, August 02, 2015 at 5:12:49 PM\n", "Khrushchev VI died on the 11th of Tuum Ut.\n", "from fire ant&y's flames!\n", "Scored &C-126&y points\n", "Survived for 1705 turns.\n", "Visited 21 zones.\n", "\tS \u0001\u001d", " \u0005 ���\u0006T �\u0001=== Game summary for &WStalin&y ===\n", "Game ended Saturday, August 01, 2015 at 3:41:26 PM\n", "Stalin died on the 1st of Tishru i Ux.\n", "from the explosion!\n", "Scored &C-224&y points\n", "Survived for 1061 turns.\n", "Visited 8 zones.\n", "\tT \u0001\u001e", " \u0005 \r", "���\u0006U �\u0001=== Game summary for &WKant VII&y ===\n", "Game ended Thursday, August 27, 2015 at 4:50:50 PM\n", "Kant VII died on the 22nd of Uulu Ut.\n", "\n", "Scored &C-243&y points\n", "Survived for 770 turns.\n", "Visited 8 zones.\n", "\tU \u0001\u001f \u0005 ����\u0006V �\u0001=== Game summary for &WKant XIV&y ===\n", "Game ended Saturday, August 29, 2015 at 12:23:09 AM\n", "Kant XIV died on the 29th of Tuum Ut.\n", "from bleeding!\n", "Scored &C-367&y points\n", "Survived for 2230 turns.\n", "Visited 14 zones.\n", "\tV \u0001 \u0005 N���\u0006W �\u0001=== Game summary for &WKant XV&y ===\n", "Game ended Saturday, August 29, 2015 at 12:42:44 AM\n", "Kant XV died on the 24th of Nivvun Ut.\n", "\n", "Scored &C-434&y points\n", "Survived for 1332 turns.\n", "Visited 10 zones.\n", "\tW \u0001! \u0005 ����\u0006X �\u0001=== Game summary for &WKhrushchev&y ===\n", "Game ended Sunday, August 02, 2015 at 2:13:37 PM\n", "Khrushchev died on the 13th of Uru Ux.\n", "from bleeding!\n", "Scored &C-531&y points\n", "Survived for 1863 turns.\n", "Visited 17 zones.\n", "\tX \u0001\" \u0005 ����\u0006Y �\u0001=== Game summary for &WNapolen III&y ===\n", "Game ended Monday, August 03, 2015 at 4:20:16 PM\n", "Napolen III died on the 20th of Shwut Ux.\n", "from the fire started by dawnglider&y!\n", "Scored &C-543&y points\n", "Survived for 1773 turns.\n", "Visited 16 zones.\n", "\tY \u0001# \u0005 ����\u0006Z �\u0001=== Game summary for &WNapoleon&y ===\n", "Game ended Monday, August 03, 2015 at 3:41:37 PM\n", "Napoleon died on the 19th of Uulu Ut.\n", "from bleeding!\n", "Scored &C-547&y points\n", "Survived for 1536 turns.\n", "Visited 11 zones.\n", "\tZ \u0001$ \u0005 ����\u0006[ �\u0001=== Game summary for &WNapoleon II&y ===\n", "Game ended Monday, August 03, 2015 at 3:57:39 PM\n", "Napoleon II died on the 22nd of Tishru i Ux.\n", "from bleeding!\n", "Scored &C-590&y points\n", "Survived for 1298 turns.\n", "Visited 11 zones.\n", "\t[ \u0001% \u0005 ����\u0006\\ �\u0001=== Game summary for &WO`Brien V&y ===\n", "Game ended Wednesday, September 02, 2015 at 12:58:51 PM\n", "O`Brien V died on the 17th of Nivvun Ut.\n", "\n", "Scored &C-618&y points\n", "Survived for 1538 turns.\n", "Visited 14 zones.\n", "\t\\ \u0001& \u0005 q���\u0006] �\u0001=== Game summary for &WKant VI&y ===\n", "Game ended Thursday, August 27, 2015 at 4:36:27 PM\n", "Kant VI died on the 14th of Shwut Ux.\n", "from &Yyoung ivory&y's impalement.\n", "Scored &C-911&y points\n", "Survived for 613 turns.\n", "Visited 7 zones.\n", "\t] \u0001' \u0005 j���\u0006^ �\u0002=== Game summary for &WKhrushchev V&y ===\n", "Game ended Sunday, August 02, 2015 at 4:45:07 PM\n", "Khrushchev V died on the 19th of Uulu Ut.\n", "The &rbloody&y salthopper&y hits &R(x4)&y for 10 damage with his &Grending mandibles ->&y10 &r\u0003&y1d4&y!&y [8]\n", "Scored &C-918&y points\n", "Survived for 510 turns.\n", "Visited 6 zones.\n", "\t^ \u0001( \u0005 X���\u0006_ �\u0002=== Game summary for &WKhrushchev IV&y ===\n", "Game ended Sunday, August 02, 2015 at 4:35:26 PM\n", "Khrushchev IV died on the 21st of Nivvun Ut.\n", "The snapjaw scavenger&y hits &w(x1)&y for 1 damage with his iron dagger ->&y2 &r\u0003&y1d4&y!&y [8]\n", "Scored &C-936&y points\n", "Survived for 1134 turns.\n", "Visited 12 zones.\n", "\t_ \u0001) \u0005 &���\u0006` �\u0002=== Game summary for &WKant II&y ===\n", "Game ended Friday, August 21, 2015 at 11:23:56 PM\n", "Kant II died on the 12th of Shwut Ux.\n", "The &cscrap shoveler&y hits &M(x5)&y for 8 damage with his &cscrap shovel ->&y15 &r\u0003&y1d2&y!&y [5]\n", "Scored &C-986&y points\n", "Survived for 720 turns.\n", "Visited 6 zones.\n", "\t` \u0001* \u0005 \u0014���\u0006a �\u0001=== Game summary for &WMalenkov&y ===\n", "Game ended Saturday, August 01, 2015 at 4:35:17 PM\n", "Malenkov died on the 14th of Tuum Ut.\n", "from snapjaw scavenger&y's explosion!\n", "Scored &C-1004&y points\n", "Survived for 911 turns.\n", "Visited 12 zones.\n", "\ta \u0001+ \u0005 ����\u0006b �\u0002=== Game summary for &WKant XVI&y ===\n", "Game ended Saturday, August 29, 2015 at 12:46:03 AM\n", "Kant XVI died on the 3rd of Uulu Ut.\n", "The &rbloody&y equimax&y hits &W(x2)&y for 5 damage with his bite ->&y9 &r\u0003&y2d2&y!&y [23]\n", "Scored &C-1120&y points\n", "Survived for 401 turns.\n", "Visited 5 zones.\n", "\tb \u0001, \u0005 ����\u0006c �\u0002=== Game summary for &WKhrushchev VII&y ===\n", "Game ended Sunday, August 02, 2015 at 5:17:56 PM\n", "Khrushchev VII died on the 27th of Uru Ux.\n", "The &rbloody&y &Rsalamander&y hits &w(x1)&y for 3 damage with his bite ->&y3 &r\u0003&y1d3&y!&y [8]\n", "Scored &C-1127&y points\n", "Survived for 403 turns.\n", "Visited 5 zones.\n", "\tc \u0001- \u0005 ����\u0006d �\u0002=== Game summary for &WStalin&y ===\n", "Game ended Saturday, August 01, 2015 at 2:04:38 PM\n", "Stalin died on the 20th of Nivvun Ut.\n", "&yUmchuum&y hits &W(x2)&y for 4 damage with his &yUmumerchacal&y!&y [9]\n", "Scored &C-1131&y points\n", "Survived for 362 turns.\n", "Visited 3 zones.\n", "Most advanced artifact in possession: &gpoison &cgas grenade mk I&y\n", "\td \u0001. \u0005 ����\u0006e �\u0001=== Game summary for &WKhrushchev IX&y ===\n", "Game ended Monday, August 03, 2015 at 1:56:40 PM\n", "Khrushchev IX died on the 18th of Tishru i Ux.\n", "from bleeding!\n", "Scored &C-1143&y points\n", "Survived for 336 turns.\n", "Visited 4 zones.\n", "\te \u0001/ \u0005 }���\u0006f �\u0002=== Game summary for &WMalenkov&y ===\n", "Game ended Saturday, August 01, 2015 at 4:18:54 PM\n", "Malenkov died on the 13th of Tishru i Ux.\n", "&MRuf-ohoubub, the stalwart Snapjaw Bear-baiter&y hits &W(x2)&y for 10 damage with his &wbronze&y two-handed sword ->&y4 &r\u0003&y1d8&y!&y [10]\n", "Scored &C-1155&y points\n", "Survived for 687 turns.\n", "Visited 8 zones.\n", "\tf \u00010 \u0005 m���\u0006g �\u0002=== Game summary for &WNapoleon IV&y ===\n", "Game ended Monday, August 03, 2015 at 4:28:33 PM\n", "Napoleon IV died on the 25th of Uulu Ut.\n", "The salthopper&y hits &W(x2)&y for 5 damage with his &Grending mandibles ->&y10 &r\u0003&y1d4&y!&y [11]\n", "Scored &C-1171&y points\n", "Survived for 275 turns.\n", "Visited 4 zones.\n", "\tg \u00011 \u0005 j���\u0006h �\u0001=== Game summary for &WO'Brien&y ===\n", "Game ended Wednesday, September 02, 2015 at 12:03:29 AM\n", "O'Brien died on the 9th of Nivvun Ut.\n", "\n", "Scored &C-1174&y points\n", "Survived for 287 turns.\n", "Visited 5 zones.\n", "\th \u00012 \u0005 i���\u0006i �\u0001=== Game summary for &WNapoleon V&y ===\n", "Game ended Monday, August 03, 2015 at 4:31:59 PM\n", "Napoleon V died on the 22nd of Ubu Ut.\n", "from bleeding!\n", "Scored &C-1175&y points\n", "Survived for 320 turns.\n", "Visited 3 zones.\n", "\ti \u00013 \u0005 G���\u0006j �\u0001=== Game summary for &WKhrushchev X&y ===\n", "Game ended Monday, August 03, 2015 at 1:59:50 PM\n", "Khrushchev X died on the 9th of Tishru ii Ux.\n", "\n", "Scored &C-1209&y points\n", "Survived for 287 turns.\n", "Visited 4 zones.\n", "\tj \u00014 \u0005 B���\u0006k �\u0002=== Game summary for &WKant II&y ===\n", "Game ended Friday, August 21, 2015 at 11:14:45 PM\n", "Kant II died on the 28th of Ubu Ut.\n", "The snapjaw hunter&y hits &r(x3)&y for 16 damage with his &wbronze&y two-handed sword ->&y4 &r\u0003&y1d8&y!&y [13]\n", "Scored &C-1214&y points\n", "Survived for 145 turns.\n", "Visited 4 zones.\n", "\tk \u00015 \u0005 \"���\u0006l �\u0002=== Game summary for &WNietzsche II&y ===\n", "Game ended Tuesday, August 04, 2015 at 8:25:57 PM\n", "Nietzsche II died on the 9th of Simmun Ut.\n", "The &rbloody&y &gjilted lover&y hits &W(x2)&y for 3 damage with his thorns ->&y5 &r\u0003&y1d4&y!&y [19]\n", "Scored &C-1246&y points\n", "Survived for 136 turns.\n", "Visited 3 zones.\n", "\tl \u00016 \u0005 \u001c", "���\u0006m �\u0001=== Game summary for &WKant III&y ===\n", "Game ended Friday, August 21, 2015 at 11:25:01 PM\n", "Kant III died on the 13th of Kisu Ux.\n", "from bleeding!\n", "Scored &C-1252&y points\n", "Survived for 105 turns.\n", "Visited 3 zones.\n", "\tm \u00017 \u0005 \u001b���\u0006n �\u0002=== Game summary for &WGoethe &y ===\n", "Game ended Sunday, August 09, 2015 at 7:43:13 PM\n", "Goethe died on the 22nd of Tuum Ut.\n", "The &rbloody&y boar&y hits &W(x2)&y for 6 damage with his bite ->&y7 &r\u0003&y1d3&y!&y [12]\n", "Scored &C-1253&y points\n", "Survived for 121 turns.\n", "Visited 3 zones.\n", "\tn \u00018 \u0005 ����\u0006o �\u0001=== Game summary for &WMalenkov&y ===\n", "Game ended Sunday, August 02, 2015 at 1:34:01 PM\n", "Malenkov died on the 9th of Tishru ii Ux.\n", "from &ctraipsing mortar&y's explosion!\n", "Scored &C-1318&y points\n", "Survived for 130 turns.\n", "Visited 5 zones.\n", "\to \u00019 \u0005 ����\u0006p �\u0001=== Game summary for &WKhrushchev III&y ===\n", "Game ended Sunday, August 02, 2015 at 4:19:46 PM\n", "Khrushchev III died on the 1st of Nivvun Ut.\n", "from the scalding steam!\n", "Scored &C-1351&y points\n", "Survived for 324 turns.\n", "Visited 4 zones.\n", "\tp \u0001: \u0005 U���\u0006q �\u0001=== Game summary for &W &y ===\n", "Game ended Friday, August 21, 2015 at 11:25:56 PM\n", " died on the 8th of Nivvun Ut.\n", "from Warden Ualraig&y's Freezes!\n", "Scored &C-1451&y points\n", "Survived for 19 turns.\n", "Visited 1 zone.\n", "\t\n", "\n", "q \u0001; \u0005 ����\u0006r �\u0001=== Game summary for &WNapoleon&y ===\n", "Game ended Monday, August 03, 2015 at 3:23:01 PM\n", "Napoleon died on the 18th of Tuum Ut.\n", "Abandoned all hope.\n", "Scored &C-1588&y points\n", "Survived for 95 turns.\n", "Visited 1 zone.\n", "\tr \u000b", "\n" ] } ], "source": [ "#This is what the HighScores file now looks like\n", "qud = open(\"HighScores.txt\", \"r\")\n", "print qud.read()\n", "qud.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There is still a number of symbols that can not be read. I have found that the following two blocks work in getting rid of these. If anyone has any better solution please suggest it." ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import codecs\n", "qud = codecs.open(\"HighScores.txt\", encoding='latin-1') #open and encode as latin-1" ] }, { "cell_type": "code", "execution_count": 42, "metadata": { "collapsed": false }, "outputs": [], "source": [ "clean_qud = open(\"HighScores_clean.txt\", \"w\") #open file to save to \n", "\n", "for line in qud: \n", " line = line.encode('utf8') \n", " line = line.decode('unicode_escape').encode('ascii','ignore')\n", " clean_qud.write(line)\n", "clean_qud.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you open both HighScores.txt and HighScores_clean.txt you will see that a number of the unreadable symbols have been removed and that the clean text is now much more readable.\n", "\n", "Next, the real cleaning begins. This will be done in two major steps. First I will completely remove all unreadable text and save this as a human readable file. Then I will use this cleaned text to create a file which will fill in missing values and add seperators between each \"column\" so it can be read into pandas.\n", "\n", "For the first step a list of tags will have to be removed such as &y, &r etc. These tags seem to determine the color of the next word or symbol on the highscores screen and all need to be removed. Also, sometimes a highscore does not contain a description of how the character died. We need to be able to determine between a blank line where this description should be and a blank line which occurs between highscores. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Game summary for Goethe II\n", "Game ended Thursday, August 13, 2015 at 6:04:58 PM\n", "Goethe II died on the 20th of Uru Ux.\n", "from Wahmahcalcalit's lase beam! (x2)\n", "Scored 48753 points\n", "Survived for 35235 turns.\n", "Visited 260 zones.\n", "Generated 1 storied items.\n", "Most advanced artifact in possession: HE Missile\n", "\n", "Game summary for Kant XVIII\n", "Game ended Sunday, August 30, 2015 at 7:34:00 PM\n", "Kant XVIII died on the 27th of Tuum Ut.\n", "The chute crab hits (x1) for 2 damage with his crab claw ->7 1d2! [7]\n", "Scored 40178 points\n", "Survived for 37145 turns.\n", "Visited 222 zones.\n", "Generated 1 storied items.\n", "Most advanced artifact in possession: Fix-It spray foam x2\n", "\n", "Game summary for O'Brien III\n", "Game ended Wednesday, September 02, 2015 at 3:50:10 AM\n", "O'Brien III died on the 6th of Tebet Ux.\n", "The bloody Kumukokumu the Stylish, legendary ogre ape hits (x8) for 51 damage with his ape fist ->20 3d3! [10]\n", "Scored 20556 points\n", "Survived for 21114 turns.\n", "Visited 130 zones.\n", "Most advanced artifact in possession: force bracelet 0 0 <> [no cell]\n", "\n", "Game summary for Kant XII\n", "Game ended Friday, August 28, 2015 at 11:14:48 PM\n", "Kant XII died on the 7th of Iyur Ut.\n", "Putus Templar warden hits (x1) for 3 damage with his folded carbide long sword ->9 2d5! [20]\n", "Scored 17061 points\n", "Survived for 17066 turns.\n", "Visited 118 zones.\n", "Most advanced artifact in possession: electrobow <> ->10 1d6 [no cell]\n", "\n", "Game summary for Nietzsche III\n", "Game ended Wednesday, August 05, 2015 at 8:00:46 PM\n", "Nietzsche III died on the 19th of Tishru ii Ux.\n", "The eyeless king crab hits (x6) for 20 damage with his massive king crab claw ->20 1d6! [11]\n", "Scored 16607 points\n", "Survived for 16124 turns.\n", "Visited 115 zones.\n", "Most advanced artifact in possession: ubernostrum injector <>\n", "\n", "Game summary for Kant XI\n", "Game ended Thursday, August 27, 2015 at 11:14:09 PM\n", "Kant XI died on the 12th of Uru Ux.\n", "from the scalding steam!\n", "Scored 9971 points\n", "Survived for 11105 turns.\n", "Visited 60 zones.\n", "Most advanced artifact in possession: Fix-It spray foam\n", "\n", "Game summary for Kant VIII\n", "Game ended Thursday, August 27, 2015 at 8:01:17 PM\n", "Kant VIII died on the 23rd of Tishru i Ux.\n", "from the scalding steam!\n", "Scored 7848 points\n", "Survived for 7719 turns.\n", "Visited 35 zones.\n", "Most advanced artifact in possession: Fix-It spray foam\n", "\n", "Game summary for Kant V\n", "Game ended Saturday, August 22, 2015 at 6:22:36 PM\n", "Kant V died on the 28th of Tishru i Ux.\n", "blank\n", "Scored 6662 points\n", "Survived for 8118 turns.\n", "Visited 47 zones.\n", "Most advanced artifact in possession: semi-automatic pistol x2 <> ->8 1d6 [Empty]\n", "\n", "Game summary for O'Brien II\n", "Game ended Wednesday, September 02, 2015 at 12:53:32 AM\n", "O'Brien II died on the 23rd of Nivvun Ut.\n", "from Duhmahcaluhcal's lase beam! (x3)\n", "Scored 5471 points\n", "Survived for 6356 turns.\n", "Visited 42 zones.\n", "Most advanced artifact in possession: stun gas grenade mk I\n", "\n", "Game summary for Kant XVII\n", "Game ended Saturday, August 29, 2015 at 1:44:21 AM\n", "Kant XVII died on the 27th of Tishru ii Ux.\n", "blank\n", "Scored 3169 points\n", "Survived for 7239 turns.\n", "Visited 37 zones.\n", "Most advanced artifact in possession: compass bracelet 0 0\n", "\n", "Game summary for Kant\n", "Game ended Friday, August 21, 2015 at 11:08:22 PM\n", "Kant died on the 12th of Tuum Ut.\n", "The giant amoeba hits (x2) for 4 damage with his giant pseudopod ->10 1d3! [15]\n", "Scored 2372 points\n", "Survived for 3013 turns.\n", "Visited 21 zones.\n", "\n", "Game summary for Kant XIII\n", "Game ended Saturday, August 29, 2015 at 12:05:24 AM\n", "Kant XIII died on the 5th of Tebet Ux.\n", "The girshling hits (x1) for 6 damage with his claw ->2 1d6! [12]\n", "Scored 1849 points\n", "Survived for 2853 turns.\n", "Visited 18 zones.\n", "Most advanced artifact in possession: pump shotgun ->8 1d2 [shotgun shell]\n", "\n", "Game summary for Khrushchev VIII\n", "Game ended Sunday, August 02, 2015 at 6:19:47 PM\n", "Khrushchev VIII died on the 5th of Tuum Ut.\n", "from bleeding!\n", "Scored 1760 points\n", "Survived for 4198 turns.\n", "Visited 25 zones.\n", "\n", "Game summary for Nietzsche\n", "Game ended Tuesday, August 04, 2015 at 8:18:00 PM\n", "Nietzsche died on the 17th of Tuum Ut.\n", "The bloody horned chameleon hits (x1) for 4 damage with his Tusks ->4 2d3 0 0! [7]\n", "Scored 1503 points\n", "Survived for 3719 turns.\n", "Visited 27 zones.\n", "\n", "Game summary for Kant IV\n", "Game ended Saturday, August 22, 2015 at 12:00:38 AM\n", "Kant IV died on the 28th of Iyur Ut.\n", "The snapjaw scavenger hits (x1) for 6 damage with his steel battle axe ->3 1d6! [18]\n", "Scored 1393 points\n", "Survived for 3347 turns.\n", "Visited 16 zones.\n", "Most advanced artifact in possession: acid gas grenade mk I\n", "\n", "Game summary for Khrushchev XI \n", "Game ended Monday, August 03, 2015 at 3:18:32 PM\n", "Khrushchev XI died on the 20th of Kisu Ux.\n", "blank\n", "Scored 1136 points\n", "Survived for 4121 turns.\n", "Visited 21 zones.\n", "Most advanced artifact in possession: blaze injector\n", "\n", "Game summary for Lenin\n", "Game ended Saturday, August 01, 2015 at 4:08:30 PM\n", "Lenin died on the 13th of Iyur Ut.\n", "The equimax hits (x3) for 9 damage with his bite ->8 2d2! [17]\n", "Scored 902 points\n", "Survived for 1019 turns.\n", "Visited 12 zones.\n", "\n", "Game summary for Khrushchev\n", "Game ended Sunday, August 02, 2015 at 3:52:52 PM\n", "Khrushchev died on the 10th of Tishru ii Ux.\n", "The bloody Groubuubu-wof-wofuz, the stalwart Snapjaw Tot-eater hits (x2) for 11 damage with his carbide battle axe ->5 2d3! [22]\n", "Scored 708 points\n", "Survived for 2789 turns.\n", "Visited 21 zones.\n", "\n", "Game summary for Kant X\n", "Game ended Thursday, August 27, 2015 at 9:17:34 PM\n", "Kant X died on the 8th of Kisu Ux.\n", "The cave spider hits (x1) for 2 damage with his fangs ->2 1d2! [19]\n", "Scored 265 points\n", "Survived for 2287 turns.\n", "Visited 16 zones.\n", "\n", "Game summary for O'Brien IV\n", "Game ended Wednesday, September 02, 2015 at 12:47:21 PM\n", "O'Brien IV died on the Ides of Uru Ux.\n", "The salthopper hits (x3) for 7 damage with his rending mandibles ->11 1d4! [15]\n", "Scored 199 points\n", "Survived for 1810 turns.\n", "Visited 14 zones.\n", "Most advanced artifact in possession: semi-automatic pistol ->8 1d6 [Empty]\n", "\n", "Game summary for Khrushchev II\n", "Game ended Sunday, August 02, 2015 at 4:13:32 PM\n", "Khrushchev II died on the 17th of Uru Ux.\n", "from bleeding!\n", "Scored 13 points\n", "Survived for 1640 turns.\n", "Visited 9 zones.\n", "\n", "Game summary for Stalin\n", "Game ended Saturday, August 01, 2015 at 3:28:05 PM\n", "Stalin died on the 27th of Uru Ux.\n", "The salthopper hits (x3) for 9 damage with his rending mandibles ->11 1d4! [16]\n", "Scored -71 points\n", "Survived for 3412 turns.\n", "Visited 30 zones.\n", "\n", "Game summary for Kant IX\n", "Game ended Thursday, August 27, 2015 at 8:51:15 PM\n", "Kant IX died on the 2nd of Iyur Ut.\n", "The jilted lover hits (x1) for 2 damage with his thorns ->5 1d4! [16]\n", "Scored -80 points\n", "Survived for 1807 turns.\n", "Visited 13 zones.\n", "Most advanced artifact in possession: acid gas grenade mk I x2\n", "\n", "Game summary for Khrushchev VI\n", "Game ended Sunday, August 02, 2015 at 5:12:49 PM\n", "Khrushchev VI died on the 11th of Tuum Ut.\n", "from fire ant's flames!\n", "Scored -126 points\n", "Survived for 1705 turns.\n", "Visited 21 zones.\n", "\n", "Game summary for Stalin\n", "Game ended Saturday, August 01, 2015 at 3:41:26 PM\n", "Stalin died on the 1st of Tishru i Ux.\n", "from the explosion!\n", "Scored -224 points\n", "Survived for 1061 turns.\n", "Visited 8 zones.\n", "\n", "Game summary for Kant VII\n", "Game ended Thursday, August 27, 2015 at 4:50:50 PM\n", "Kant VII died on the 22nd of Uulu Ut.\n", "blank\n", "Scored -243 points\n", "Survived for 770 turns.\n", "Visited 8 zones.\n", "\n", "Game summary for Kant XIV\n", "Game ended Saturday, August 29, 2015 at 12:23:09 AM\n", "Kant XIV died on the 29th of Tuum Ut.\n", "from bleeding!\n", "Scored -367 points\n", "Survived for 2230 turns.\n", "Visited 14 zones.\n", "\n", "Game summary for Kant XV\n", "Game ended Saturday, August 29, 2015 at 12:42:44 AM\n", "Kant XV died on the 24th of Nivvun Ut.\n", "blank\n", "Scored -434 points\n", "Survived for 1332 turns.\n", "Visited 10 zones.\n", "\n", "Game summary for Khrushchev\n", "Game ended Sunday, August 02, 2015 at 2:13:37 PM\n", "Khrushchev died on the 13th of Uru Ux.\n", "from bleeding!\n", "Scored -531 points\n", "Survived for 1863 turns.\n", "Visited 17 zones.\n", "\n", "Game summary for Napolen III\n", "Game ended Monday, August 03, 2015 at 4:20:16 PM\n", "Napolen III died on the 20th of Shwut Ux.\n", "from the fire started by dawnglider!\n", "Scored -543 points\n", "Survived for 1773 turns.\n", "Visited 16 zones.\n", "\n", "Game summary for Napoleon\n", "Game ended Monday, August 03, 2015 at 3:41:37 PM\n", "Napoleon died on the 19th of Uulu Ut.\n", "from bleeding!\n", "Scored -547 points\n", "Survived for 1536 turns.\n", "Visited 11 zones.\n", "\n", "Game summary for Napoleon II\n", "Game ended Monday, August 03, 2015 at 3:57:39 PM\n", "Napoleon II died on the 22nd of Tishru i Ux.\n", "from bleeding!\n", "Scored -590 points\n", "Survived for 1298 turns.\n", "Visited 11 zones.\n", "\n", "Game summary for O'Brien V\n", "Game ended Wednesday, September 02, 2015 at 12:58:51 PM\n", "O'Brien V died on the 17th of Nivvun Ut.\n", "blank\n", "Scored -618 points\n", "Survived for 1538 turns.\n", "Visited 14 zones.\n", "\n", "Game summary for Kant VI\n", "Game ended Thursday, August 27, 2015 at 4:36:27 PM\n", "Kant VI died on the 14th of Shwut Ux.\n", "from young ivory's impalement.\n", "Scored -911 points\n", "Survived for 613 turns.\n", "Visited 7 zones.\n", "\n", "Game summary for Khrushchev V\n", "Game ended Sunday, August 02, 2015 at 4:45:07 PM\n", "Khrushchev V died on the 19th of Uulu Ut.\n", "The bloody salthopper hits (x4) for 10 damage with his rending mandibles ->10 1d4! [8]\n", "Scored -918 points\n", "Survived for 510 turns.\n", "Visited 6 zones.\n", "\n", "Game summary for Khrushchev IV\n", "Game ended Sunday, August 02, 2015 at 4:35:26 PM\n", "Khrushchev IV died on the 21st of Nivvun Ut.\n", "The snapjaw scavenger hits (x1) for 1 damage with his iron dagger ->2 1d4! [8]\n", "Scored -936 points\n", "Survived for 1134 turns.\n", "Visited 12 zones.\n", "\n", "Game summary for Kant II\n", "Game ended Friday, August 21, 2015 at 11:23:56 PM\n", "Kant II died on the 12th of Shwut Ux.\n", "The scrap shoveler hits (x5) for 8 damage with his scrap shovel ->15 1d2! [5]\n", "Scored -986 points\n", "Survived for 720 turns.\n", "Visited 6 zones.\n", "\n", "Game summary for Malenkov\n", "Game ended Saturday, August 01, 2015 at 4:35:17 PM\n", "Malenkov died on the 14th of Tuum Ut.\n", "from snapjaw scavenger's explosion!\n", "Scored -1004 points\n", "Survived for 911 turns.\n", "Visited 12 zones.\n", "\n", "Game summary for Kant XVI\n", "Game ended Saturday, August 29, 2015 at 12:46:03 AM\n", "Kant XVI died on the 3rd of Uulu Ut.\n", "The bloody equimax hits (x2) for 5 damage with his bite ->9 2d2! [23]\n", "Scored -1120 points\n", "Survived for 401 turns.\n", "Visited 5 zones.\n", "\n", "Game summary for Khrushchev VII\n", "Game ended Sunday, August 02, 2015 at 5:17:56 PM\n", "Khrushchev VII died on the 27th of Uru Ux.\n", "The bloody salamander hits (x1) for 3 damage with his bite ->3 1d3! [8]\n", "Scored -1127 points\n", "Survived for 403 turns.\n", "Visited 5 zones.\n", "\n", "Game summary for Stalin\n", "Game ended Saturday, August 01, 2015 at 2:04:38 PM\n", "Stalin died on the 20th of Nivvun Ut.\n", "Umchuum hits (x2) for 4 damage with his Umumerchacal! [9]\n", "Scored -1131 points\n", "Survived for 362 turns.\n", "Visited 3 zones.\n", "Most advanced artifact in possession: poison gas grenade mk I\n", "\n", "Game summary for Khrushchev IX\n", "Game ended Monday, August 03, 2015 at 1:56:40 PM\n", "Khrushchev IX died on the 18th of Tishru i Ux.\n", "from bleeding!\n", "Scored -1143 points\n", "Survived for 336 turns.\n", "Visited 4 zones.\n", "\n", "Game summary for Malenkov\n", "Game ended Saturday, August 01, 2015 at 4:18:54 PM\n", "Malenkov died on the 13th of Tishru i Ux.\n", "Ruf-ohoubub, the stalwart Snapjaw Bear-baiter hits (x2) for 10 damage with his bronze two-handed sword ->4 1d8! [10]\n", "Scored -1155 points\n", "Survived for 687 turns.\n", "Visited 8 zones.\n", "\n", "Game summary for Napoleon IV\n", "Game ended Monday, August 03, 2015 at 4:28:33 PM\n", "Napoleon IV died on the 25th of Uulu Ut.\n", "The salthopper hits (x2) for 5 damage with his rending mandibles ->10 1d4! [11]\n", "Scored -1171 points\n", "Survived for 275 turns.\n", "Visited 4 zones.\n", "\n", "Game summary for O'Brien\n", "Game ended Wednesday, September 02, 2015 at 12:03:29 AM\n", "O'Brien died on the 9th of Nivvun Ut.\n", "blank\n", "Scored -1174 points\n", "Survived for 287 turns.\n", "Visited 5 zones.\n", "\n", "Game summary for Napoleon V\n", "Game ended Monday, August 03, 2015 at 4:31:59 PM\n", "Napoleon V died on the 22nd of Ubu Ut.\n", "from bleeding!\n", "Scored -1175 points\n", "Survived for 320 turns.\n", "Visited 3 zones.\n", "\n", "Game summary for Khrushchev X\n", "Game ended Monday, August 03, 2015 at 1:59:50 PM\n", "Khrushchev X died on the 9th of Tishru ii Ux.\n", "blank\n", "Scored -1209 points\n", "Survived for 287 turns.\n", "Visited 4 zones.\n", "\n", "Game summary for Kant II\n", "Game ended Friday, August 21, 2015 at 11:14:45 PM\n", "Kant II died on the 28th of Ubu Ut.\n", "The snapjaw hunter hits (x3) for 16 damage with his bronze two-handed sword ->4 1d8! [13]\n", "Scored -1214 points\n", "Survived for 145 turns.\n", "Visited 4 zones.\n", "\n", "Game summary for Nietzsche II\n", "Game ended Tuesday, August 04, 2015 at 8:25:57 PM\n", "Nietzsche II died on the 9th of Simmun Ut.\n", "The bloody jilted lover hits (x2) for 3 damage with his thorns ->5 1d4! [19]\n", "Scored -1246 points\n", "Survived for 136 turns.\n", "Visited 3 zones.\n", "\n", "Game summary for Kant III\n", "Game ended Friday, August 21, 2015 at 11:25:01 PM\n", "Kant III died on the 13th of Kisu Ux.\n", "from bleeding!\n", "Scored -1252 points\n", "Survived for 105 turns.\n", "Visited 3 zones.\n", "\n", "Game summary for Goethe \n", "Game ended Sunday, August 09, 2015 at 7:43:13 PM\n", "Goethe died on the 22nd of Tuum Ut.\n", "The bloody boar hits (x2) for 6 damage with his bite ->7 1d3! [12]\n", "Scored -1253 points\n", "Survived for 121 turns.\n", "Visited 3 zones.\n", "\n", "Game summary for Malenkov\n", "Game ended Sunday, August 02, 2015 at 1:34:01 PM\n", "Malenkov died on the 9th of Tishru ii Ux.\n", "from traipsing mortar's explosion!\n", "Scored -1318 points\n", "Survived for 130 turns.\n", "Visited 5 zones.\n", "\n", "Game summary for Khrushchev III\n", "Game ended Sunday, August 02, 2015 at 4:19:46 PM\n", "Khrushchev III died on the 1st of Nivvun Ut.\n", "from the scalding steam!\n", "Scored -1351 points\n", "Survived for 324 turns.\n", "Visited 4 zones.\n", "\n", "Game summary for \n", "Game ended Friday, August 21, 2015 at 11:25:56 PM\n", "died on the 8th of Nivvun Ut.\n", "from Warden Ualraig's Freezes!\n", "Scored -1451 points\n", "Survived for 19 turns.\n", "Visited 1 zone.\n", "\n", "Game summary for Napoleon\n", "Game ended Monday, August 03, 2015 at 3:23:01 PM\n", "Napoleon died on the 18th of Tuum Ut.\n", "Abandoned all hope.\n", "Scored -1588 points\n", "Survived for 95 turns.\n", "Visited 1 zone.\n" ] } ], "source": [ "import re\n", "\n", "#list of text to remove.\n", "remove_list = [\"&W\", \"&y\", \"&w\", \"&r\", \"&M\", \"&Y\", \"&C\", \"&c\", \"&b\", \"&B\", \"&K\", \"&R\", \"&W\", \"&G\", \"&g\", \"\\r\", \"\\n\", \"\\t\"]\n", "\n", "cleaned_qud = open(\"Cleaned_Qud_HighScores.txt\", \"w\") #file to write to\n", "\n", "clean_highscores = open(\"HighScores_clean.txt\", \"r\")\n", "\n", "#flag which will be used to determine if there is a blank line in the data instead of a line describing how the character died.\n", "#this represents if we have reached a line that says \"Visited x Zones\" which is always present and always occurs after the character death description\n", "visited = False\n", "#flag which will be used to determine if this is the first line in the file\n", "first_line = True\n", "\n", "for line in clean_highscores.readlines():\n", " line = line.replace(\"`\", \"'\").replace(\"\\n\", \" \").strip() #some lines have a different ' which was causing havoc! Remove all linebreaks, strip away all whitespace\n", " \n", " for remove_word in remove_list:\n", " line = line.replace(remove_word, \"\") #go down through all words in the remove list and replace them with \"\"\n", " if \"===\" in line: #check if this is the first line of a highscore (===Game summary for )\n", " visited = False #set the visited variable to false\n", " qud_search = re.search(\"=== ((\\w*\\'*\\w*\\s*)*) ===\", line) #pull out everything between === and ===\n", " if first_line == True:\n", " line = str(qud_search.group(1))\n", " first_line = False\n", " else:\n", " line = \"\\n\" + str(qud_search.group(1)) #if this is the first line in the file write as is, otherwise put a \\n at the start. Prevents a blank line at the start of the file\n", " \n", " if len(line) == 0 and visited == False: #if a line is blank and we haven't hit the end of the highscore this is where a death description should be\n", " line = \"blank\"\n", " \n", " if \"Visited\" in line:\n", " visited = True #set to true to indicate we have passed the death description. Blank lines after this will be striped out\n", " \n", " #Even after all the cleaning some unwanted symbols were still getting through. The following line works, but is messy. But works. Did I mention it works?...well, it works so far...\n", " #If we have passed the death description (visited == True) any file striped of ALL spaces, even those between words, that is less than 10 letters can be assumed to be trash that has made it through the cleaning process. Delete.\n", " if visited == True:\n", " if len(line.replace(\" \", \"\")) < 10:\n", " #continue \n", " \n", " print line #print out the line\n", " cleaned_qud.write(line + \"\\n\") #write the line\n", "cleaned_qud.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Wow, that was tough and we're still not near Golgotha. This human readable file created above will now be used to create a pandas readable file. This could have all been done in one step but is done in two for my sanity, which I was in danger of losing during the above process and also in the event a user would rather change the below step to clean the file in a different way.\n", "\n", "Now we need to delete a lot of the filler text (\"Game summary for \", \"x died on the \" etc) so that we are just left with catagorial (character name, artifact name) or integer values (score, zones).\n", "\n", "There is also the issue of uneven or unequal highscore descriptions. Some of them contain data that the others do not. If I found a \"storied item\" (I remember finding a shield called \"Stopslavin\") then a row \"Generated 1 storied items.\" will be added. However, if I do not find a storied item then this line will not be there. Same with artifacts. So it is possible that some scores will have (at least) two lines more than other scores and a number of flags are used to check this.\n", "\n", "If going through this code using your own highscores data you will more than likely have to make adjustments/additions to the lines determining how the character died." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Goethe II\n", "Thursday, August 13, 2015 6:04:58 PM\n", "20th of Uru Ux\n", "Wahmahcalcalit\t0\t0\tlase beam\t0\t0\t\n", "48753\n", "35235\n", "260\n", "1\n", "HE Missile\n", "\n", "\n", "Kant XVIII\n", "Sunday, August 30, 2015 7:34:00 PM\n", "27th of Tuum Ut\n", "chute crab\t1\t2\tcrab claw\t7\t1d2\n", "40178\n", "37145\n", "222\n", "1\n", "Fix-It spray foam x2\n", "\n", "\n", "O'Brien III\n", "Wednesday, September 02, 2015 3:50:10 AM\n", "6th of Tebet Ux\n", "Kumukokumu the Stylish, legendary ogre ape\t8\t51\tape fist \t20\t3d3\n", "20556\n", "21114\n", "130\n", "0\tforce bracelet 0 0 <> [no cell]\n", "\n", "\n", "Kant XII\n", "Friday, August 28, 2015 11:14:48 PM\n", "7th of Iyur Ut\n", "Putus Templar warden\t1\t3\tfolded carbide long sword\t9\t2d5\n", "17061\n", "17066\n", "118\n", "0\telectrobow <> ->10 1d6 [no cell]\n", "\n", "\n", "Nietzsche III\n", "Wednesday, August 05, 2015 8:00:46 PM\n", "19th of Tishru ii Ux\n", "eyeless king crab\t6\t20\tmassive king crab claw\t20\t1d6\n", "16607\n", "16124\n", "115\n", "0\tubernostrum injector <>\n", "\n", "\n", "Kant XI\n", "Thursday, August 27, 2015 11:14:09 PM\n", "12th of Uru Ux\n", "scalding steam\t0\t0\tscalding steam\t0\t0\n", "9971\n", "11105\n", "60\n", "0\tFix-It spray foam\n", "\n", "\n", "Kant VIII\n", "Thursday, August 27, 2015 8:01:17 PM\n", "23rd of Tishru i Ux\n", "scalding steam\t0\t0\tscalding steam\t0\t0\n", "7848\n", "7719\n", "35\n", "0\tFix-It spray foam\n", "\n", "\n", "Kant V\n", "Surday, August 22, 2015 6:22:36 PM\n", "28th of Tishru i Ux\n", "unknown\t0\t0\tunknown\t0\t0\n", "6662\n", "8118\n", "47\n", "0\tsemi-automatic pistol x2 <> ->8 1d6 [Empty]\n", "\n", "\n", "O'Brien II\n", "Wednesday, September 02, 2015 12:53:32 AM\n", "23rd of Nivvun Ut\n", "Duhmahcaluhcal\t0\t0\tlase beam\t0\t0\t\n", "5471\n", "6356\n", "42\n", "0\tstun gas grenade mk I\n", "\n", "\n", "Kant XVII\n", "Surday, August 29, 2015 1:44:21 AM\n", "27th of Tishru ii Ux\n", "unknown\t0\t0\tunknown\t0\t0\n", "3169\n", "7239\n", "37\n", "0\tcompass bracelet 0 0\n", "\n", "\n", "Kant\n", "Friday, August 21, 2015 11:08:22 PM\n", "12th of Tuum Ut\n", "giant amoeba\t2\t4\tgiant pseudopod\t10\t1d3\n", "2372\n", "3013\n", "21\n", "0\tno artifact\t\n", "\n", "Kant XIII\n", "Surday, August 29, 2015 12:05:24 AM\n", "5th of Tebet Ux\n", "girshling\t1\t6\tclaw\t2\t1d6\n", "1849\n", "2853\n", "18\n", "0\tpump shotgun ->8 1d2 [shotgun shell]\n", "\n", "\n", "Khrushchev VIII\n", "Sunday, August 02, 2015 6:19:47 PM\n", "5th of Tuum Ut\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "1760\n", "4198\n", "25\n", "0\tno artifact\t\n", "\n", "Nietzsche\n", "Tuesday, August 04, 2015 8:18:00 PM\n", "17th of Tuum Ut\n", "horned chameleon\t1\t4\tTusks\t4\t2d3\n", "1503\n", "3719\n", "27\n", "0\tno artifact\t\n", "\n", "Kant IV\n", "Surday, August 22, 2015 12:00:38 AM\n", "28th of Iyur Ut\n", "snapjaw scavenger\t1\t6\tsteel battle axe\t3\t1d6\n", "1393\n", "3347\n", "16\n", "0\tacid gas grenade mk I\n", "\n", "\n", "Khrushchev XI\n", "Monday, August 03, 2015 3:18:32 PM\n", "Khrushchev XI died on the 20th of Kisu Ux\n", "unknown\t0\t0\tunknown\t0\t0\n", "1136\n", "4121\n", "21\n", "0\tblaze injector\n", "\n", "\n", "Lenin\n", "Surday, August 01, 2015 4:08:30 PM\n", "13th of Iyur Ut\n", "equimax\t3\t9\tbite\t8\t2d2\n", "902\n", "1019\n", "12\n", "0\tno artifact\t\n", "\n", "Khrushchev\n", "Sunday, August 02, 2015 3:52:52 PM\n", "10th of Tishru ii Ux\n", "Groubuubu-wof-wofuz, the stalwart Snapjaw Tot-eater\t2\t11\tcarbide battle axe\t5\t2d3\n", "708\n", "2789\n", "21\n", "0\tno artifact\t\n", "\n", "Kant X\n", "Thursday, August 27, 2015 9:17:34 PM\n", "8th of Kisu Ux\n", "cave spider\t1\t2\tfangs\t2\t1d2\n", "265\n", "2287\n", "16\n", "0\tno artifact\t\n", "\n", "O'Brien IV\n", "Wednesday, September 02, 2015 12:47:21 PM\n", "Ides of Uru Ux\n", "salthopper\t3\t7\trending mandibles\t11\t1d4\n", "199\n", "1810\n", "14\n", "0\tsemi-automatic pistol ->8 1d6 [Empty]\n", "\n", "\n", "Khrushchev II\n", "Sunday, August 02, 2015 4:13:32 PM\n", "17th of Uru Ux\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "13\n", "1640\n", "9\n", "0\tno artifact\t\n", "\n", "Stalin\n", "Surday, August 01, 2015 3:28:05 PM\n", "27th of Uru Ux\n", "salthopper\t3\t9\trending mandibles\t11\t1d4\n", "-71\n", "3412\n", "30\n", "0\tno artifact\t\n", "\n", "Kant IX\n", "Thursday, August 27, 2015 8:51:15 PM\n", "2nd of Iyur Ut\n", "jilted lover\t1\t2\tthorns\t5\t1d4\n", "-80\n", "1807\n", "13\n", "0\tacid gas grenade mk I x2\n", "\n", "\n", "Khrushchev VI\n", "Sunday, August 02, 2015 5:12:49 PM\n", "11th of Tuum Ut\n", "fire ant\t0\t0\tflames\t0\t0\t\n", "-126\n", "1705\n", "21\n", "0\tno artifact\t\n", "\n", "Stalin\n", "Surday, August 01, 2015 3:41:26 PM\n", "1st of Tishru i Ux\n", "explosion\t0\t0\texplosion\t0\t0\n", "-224\n", "1061\n", "8\n", "0\tno artifact\t\n", "\n", "Kant VII\n", "Thursday, August 27, 2015 4:50:50 PM\n", "22nd of Uulu Ut\n", "unknown\t0\t0\tunknown\t0\t0\n", "-243\n", "770\n", "8\n", "0\tno artifact\t\n", "\n", "Kant XIV\n", "Surday, August 29, 2015 12:23:09 AM\n", "29th of Tuum Ut\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "-367\n", "2230\n", "14\n", "0\tno artifact\t\n", "\n", "Kant XV\n", "Surday, August 29, 2015 12:42:44 AM\n", "24th of Nivvun Ut\n", "unknown\t0\t0\tunknown\t0\t0\n", "-434\n", "1332\n", "10\n", "0\tno artifact\t\n", "\n", "Khrushchev\n", "Sunday, August 02, 2015 2:13:37 PM\n", "13th of Uru Ux\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "-531\n", "1863\n", "17\n", "0\tno artifact\t\n", "\n", "Napolen III\n", "Monday, August 03, 2015 4:20:16 PM\n", "20th of Shwut Ux\n", "dawnglider\t0\t0\tfire\t0\t0\n", "-543\n", "1773\n", "16\n", "0\tno artifact\t\n", "\n", "Napoleon\n", "Monday, August 03, 2015 3:41:37 PM\n", "19th of Uulu Ut\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "-547\n", "1536\n", "11\n", "0\tno artifact\t\n", "\n", "Napoleon II\n", "Monday, August 03, 2015 3:57:39 PM\n", "22nd of Tishru i Ux\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "-590\n", "1298\n", "11\n", "0\tno artifact\t\n", "\n", "O'Brien V\n", "Wednesday, September 02, 2015 12:58:51 PM\n", "17th of Nivvun Ut\n", "unknown\t0\t0\tunknown\t0\t0\n", "-618\n", "1538\n", "14\n", "0\tno artifact\t\n", "\n", "Kant VI\n", "Thursday, August 27, 2015 4:36:27 PM\n", "14th of Shwut Ux\n", "young ivory\t0\t0\timpalement\t0\t0\t\n", "-911\n", "613\n", "7\n", "0\tno artifact\t\n", "\n", "Khrushchev V\n", "Sunday, August 02, 2015 4:45:07 PM\n", "19th of Uulu Ut\n", "salthopper\t4\t10\trending mandibles\t10\t1d4\n", "-918\n", "510\n", "6\n", "0\tno artifact\t\n", "\n", "Khrushchev IV\n", "Sunday, August 02, 2015 4:35:26 PM\n", "21st of Nivvun Ut\n", "snapjaw scavenger\t1\t1\tiron dagger\t2\t1d4\n", "-936\n", "1134\n", "12\n", "0\tno artifact\t\n", "\n", "Kant II\n", "Friday, August 21, 2015 11:23:56 PM\n", "12th of Shwut Ux\n", "scrap shoveler\t5\t8\tscrap shovel\t15\t1d2\n", "-986\n", "720\n", "6\n", "0\tno artifact\t\n", "\n", "Malenkov\n", "Surday, August 01, 2015 4:35:17 PM\n", "14th of Tuum Ut\n", "napjaw scavenger\t0\t0\texplosion\t0\t0\t\n", "-1004\n", "911\n", "12\n", "0\tno artifact\t\n", "\n", "Kant XVI\n", "Surday, August 29, 2015 12:46:03 AM\n", "3rd of Uulu Ut\n", "equimax\t2\t5\tbite\t9\t2d2\n", "-1120\n", "401\n", "5\n", "0\tno artifact\t\n", "\n", "Khrushchev VII\n", "Sunday, August 02, 2015 5:17:56 PM\n", "27th of Uru Ux\n", "salamander\t1\t3\tbite\t3\t1d3\n", "-1127\n", "403\n", "5\n", "0\tno artifact\t\n", "\n", "Stalin\n", "Surday, August 01, 2015 2:04:38 PM\n", "20th of Nivvun Ut\n", "Umchuum\t2\t4\tUmumerchacal\t0\t0\n", "-1131\n", "362\n", "3\n", "0\tpoison gas grenade mk I\n", "\n", "\n", "Khrushchev IX\n", "Monday, August 03, 2015 1:56:40 PM\n", "18th of Tishru i Ux\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "-1143\n", "336\n", "4\n", "0\tno artifact\t\n", "\n", "Malenkov\n", "Surday, August 01, 2015 4:18:54 PM\n", "13th of Tishru i Ux\n", "Ruf-ohoubub, the stalwart Snapjaw Bear-baiter\t2\t10\tbronze two-handed sword\t4\t1d8\n", "-1155\n", "687\n", "8\n", "0\tno artifact\t\n", "\n", "Napoleon IV\n", "Monday, August 03, 2015 4:28:33 PM\n", "25th of Uulu Ut\n", "salthopper\t2\t5\trending mandibles\t10\t1d4\n", "-1171\n", "275\n", "4\n", "0\tno artifact\t\n", "\n", "O'Brien\n", "Wednesday, September 02, 2015 12:03:29 AM\n", "9th of Nivvun Ut\n", "unknown\t0\t0\tunknown\t0\t0\n", "-1174\n", "287\n", "5\n", "0\tno artifact\t\n", "\n", "Napoleon V\n", "Monday, August 03, 2015 4:31:59 PM\n", "22nd of Ubu Ut\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "-1175\n", "320\n", "3\n", "0\tno artifact\t\n", "\n", "Khrushchev X\n", "Monday, August 03, 2015 1:59:50 PM\n", "9th of Tishru ii Ux\n", "unknown\t0\t0\tunknown\t0\t0\n", "-1209\n", "287\n", "4\n", "0\tno artifact\t\n", "\n", "Kant II\n", "Friday, August 21, 2015 11:14:45 PM\n", "28th of Ubu Ut\n", "snapjaw hunter\t3\t16\tbronze two-handed sword\t4\t1d8\n", "-1214\n", "145\n", "4\n", "0\tno artifact\t\n", "\n", "Nietzsche II\n", "Tuesday, August 04, 2015 8:25:57 PM\n", "9th of Simmun Ut\n", "jilted lover\t2\t3\tthorns\t5\t1d4\n", "-1246\n", "136\n", "3\n", "0\tno artifact\t\n", "\n", "Kant III\n", "Friday, August 21, 2015 11:25:01 PM\n", "13th of Kisu Ux\n", "bleeding\t0\t0\tbleeding\t0\t0\n", "-1252\n", "105\n", "3\n", "0\tno artifact\t\n", "\n", "Goethe\n", "Sunday, August 09, 2015 7:43:13 PM\n", "Goethe died on the 22nd of Tuum Ut\n", "boar\t2\t6\tbite\t7\t1d3\n", "-1253\n", "121\n", "3\n", "0\tno artifact\t\n", "\n", "Malenkov\n", "Sunday, August 02, 2015 1:34:01 PM\n", "9th of Tishru ii Ux\n", "traipsing mortar\t0\t0\texplosion\t0\t0\t\n", "-1318\n", "130\n", "5\n", "0\tno artifact\t\n", "\n", "Khrushchev III\n", "Sunday, August 02, 2015 4:19:46 PM\n", "1st of Nivvun Ut\n", "scalding steam\t0\t0\tscalding steam\t0\t0\n", "-1351\n", "324\n", "4\n", "0\tno artifact\t\n", "\n", "Game summary for\n", "Friday, August 21, 2015 11:25:56 PM\n", "died on the 8th of Nivvun Ut\n", "Warden Ualraig\t0\t0\tFreezes\t0\t0\t\n", "-1451\n", "19\n", "1\n", "0\tno artifact\t\n", "\n", "Napoleon\n", "Monday, August 03, 2015 3:23:01 PM\n", "18th of Tuum Ut\n", "quit\t0\t0\tquit\t0\t0\n", "-1588\n", "95\n", "1\n" ] } ], "source": [ "import re #left behind as I often started the the notebook from this point, content with the cleaning in the above step from earlier\n", "\n", "cleaned_qud = open(\"Cleaned_Qud_HighScores_1.txt\", \"w\")\n", "\n", "clean_highscores = open(\"Cleaned_Qud_HighScores.txt\", \"r\")\n", "\n", "first_line = True\n", "name = \" \"\n", "#flags for checking if storied items or artifacts are present in the highscore\n", "visited = False\n", "generated = False\n", "artifact = False\n", "\n", "for line in clean_highscores.readlines():\n", " \n", " line = line.replace(\"`\", \"'\").replace(\"\\n\", \" \").replace(\".\", \"\").strip() \n", " \n", " if \"summary\" in line: #If this is the first line of a highscore\n", " visited = False\n", " generated = False #set all flags to false\n", " artifact = False\n", " line = line.replace(\"Game summary for \", \"\") #remove everything but the characters name\n", " name = line #save the characters name to be used in a later deletion (\"name died on \")\n", " line = line.strip() #strip blank space. This is from an attempt to parse a line where the character name was \" \"\n", " if first_line == True:\n", " first_line = False\n", " else:\n", " line = \"\\n\"+str(line) #If this is the first line saved to the file add as is, other wise add a \\n to the start \n", " \n", " if \"Game ended\" in line:\n", " line = line.replace(\"Game ended\", \"\").replace(\"at\", \"\").strip() #Remove \"Game ended\", leaving behind only the date \n", " \n", " \n", " if \"died on\" in line:\n", " line = line.replace(\"%s died on the\" % name, \"\").strip() #remove \"name died on the \" leaving behind only the Game date\n", " \n", " \n", " #Code to figure out what caused the players death\n", " if \" hits (\" in line:\n", " #The chute crab hits (x1) for 2 damage with his crab claw ->7 1d2! [7]\n", " if \"->\" in line:\n", " death_search = re.search(\"((\\w*\\,?\\-?\\s*)+) hits \\(x(\\d*)\\) for (\\d*) damage with \\w{3} ((\\w*\\,?\\-?\\s*)+) ->(\\d+) (\\d*d\\d*)!?\", line.replace(\"The\", \"\").replace(\"bloody\", \"\").strip())\n", " #name, times hit, damage, weapon, PV, pos damage\n", " line = str(death_search.group(1)) + \"\\t\" + str(death_search.group(3)) + \"\\t\" + str(death_search.group(4)) + \"\\t\" + str(death_search.group(5)) + \"\\t\" + str(death_search.group(7)) + \"\\t\" + str(death_search.group(8)) \n", " else:\n", " #Umchuum hits (x2) for 4 damage with his Umumerchacal! [9]\n", " death_search = re.search(\"((\\w*\\,?\\-?\\s*)+) hits \\(x(\\d*)\\) for (\\d*) damage with \\w{3} ((\\w*\\,?\\-?\\s*)+)!?\", line.replace(\"The\", \"\").replace(\"bloody\", \"\").strip())\n", " #name, times hit, damage, weapon, PV, pos damage\n", " line = str(death_search.group(1)) + \"\\t\" + str(death_search.group(3)) + \"\\t\" + str(death_search.group(4)) + \"\\t\" + str(death_search.group(5)) + \"\\t0\" + \"\\t0\" \n", "\n", " if \"blank\" in line:\n", " line = \"unknown\\t0\\t0\\tunknown\\t0\\t0\"\n", " \n", " #lines that contain 'from' are generally short descriptions. A more effective regex could be written at a later time.\n", " if \"from\" in line:\n", " if line.strip() == \"from bleeding!\":\n", " line = \"bleeding\\t0\\t0\\tbleeding\\t0\\t0\"\n", " \n", " elif line.strip() == \"from the scalding steam!\":\n", " line = \"scalding steam\\t0\\t0\\tscalding steam\\t0\\t0\"\n", " \n", " elif line.strip() == \"from the explosion!\":\n", " line = \"explosion\\t0\\t0\\texplosion\\t0\\t0\"\n", " \n", " elif \"from the fire started by\" in line:\n", " foe = line.replace(\"from the fire started by \", \"\").strip(\"!\").strip()\n", " line = \"%s\\t0\\t0\\tfire\\t0\\t0\" % foe\n", " \n", " elif \"'s\" in line:\n", " #from Wahmahcalcalit's lase beam!\n", " death_search = re.search(\"from ((\\w*\\,?\\-?\\s*)*(\\w*\\,?\\-?(\\'s){1}\\s*)) ((\\w*\\,?\\-?(\\'s){0}\\s*)+)\", line)\n", " line = \"%s\\t0\\t0\\t%s\\t0\\t0\\t\" % (str(death_search.group(1).strip(\"'s'\")), str(death_search.group(5))) \n", " \n", " \n", " if line.strip() == \"Abandoned all hope\":\n", " line = \"quit\\t0\\t0\\tquit\\t0\\t0\"\n", " \n", " \n", " if \"Scored\" in line:\n", " line = line.replace(\"Scored\", \"\").replace(\"points\", \"\").strip() #remove all bar the points figure\n", " \n", " \n", "\n", " if \"Survived \" in line:\n", " line = line.replace(\"Survived for\", \"\").replace(\"turns\", \"\").strip() #remove all bar the turns figure\n", " \n", "\n", "\n", " if \"Visited \" in line:\n", " visited = True #set visited flag to true\n", " line = line.replace(\"Visited\", \"\").replace(\"zones\", \"\").replace(\"zone\", \"\").strip() #remove all bar the zones figure\n", " \n", " \n", " \n", " \n", " if \"Generated\" in line:\n", " generated = True #set generated flag to true\n", " line = line.replace(\"Generated\", \"\").replace(\"storied items\", \"\").strip() #remove all bar the storied items figure\n", " \n", " \n", " \n", " \n", "\n", " if \"Most advanced artifact\" in line:\n", " artifact = True #set artifact flag to true\n", " gen_check = \"\" #create a string for checking if a storied items figure exists\n", " if generated == False: \n", " gen_check = \"0\\t\"\n", " generated = True\n", " line = gen_check + str(line.replace(\"Most advanced artifact in possession:\", \"\").strip())\n", " #if there is an artifact but no storied item this will read \"0\\t\" + artifactname. If there is a storied item this will be \"\" + artifactname\n", " \n", " \n", " \n", " if len(line) == 0 and visited == True: #if we are on a blank line and we have passed the visited line this will add \"0 no artifact\" to the end of the line\n", " if generated == False:\n", " line = str(line) + \"0\\t\" \n", " \n", " if artifact == False:\n", " line = str(line) + \"no artifact\\t\" \n", " \n", " \n", " \n", " print line\n", " cleaned_qud.write(line + \"\\t\")\n", "cleaned_qud.write(\"0\\tno artifact\") #insert into final row \n", "cleaned_qud.close()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The text is now cleaned and can be read into a pandas dataframe. The above code works with my current highscores but a lot of work would need to be done to make it compatiable with other players highscores. There are many ways to die in Qud and my parsing only takes into consideration the few ways my characters have died. I am sure there are many ways to break the above code and I would greatly appreciate any suggestions or improvements." ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.8" } }, "nbformat": 4, "nbformat_minor": 0 }