{ "cells": [ { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 1 Business Problem" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "The Canadian banking system continues to rank at the top of the world thanks to the continuous effort to improve our quality control practices. As evident during the 2008 Sub-Prime Mortgage Crisis, Canada was one of the few countries that withstood the Great Recession.\n", "\n", "One approach to improve quality control practices is by analyzing the quality of a Bank's business portfolio for each individual business line. For example, a Bank's core business line could be providing construction loan products, and based on the rationale behind each deal for the approval and denial of construction loans, we can also determine the topics in each decision from the rationales. By determining the topics in each decision, we can then perform quality control to ensure all the decisions that were made are in accordance to the Bank's risk appetite and pricing.\n", "\n", "With this approach, Banks can improve the quality of their construction loan business from their own decision making standards, and thus improving the overall quality of their business.\n", "\n", "However, in order to get this information, the Bank needs to extract topics from hundreds and thousands of data, and then interpret the topics before determining if the decisions that were made meets the Bank's decision making standards, all of which can take a lot of time and resources to complete.\n", "
\n", "
\n", "\n", "**Business Solutions:**\n", "\n", "To solve this issue, I have created a \"Quality Control System\" that learns and extracts topics from a Bank's rationale for decision making. This can then be used as quality control to determine if the decisions that were made are in accordance to the Bank's standards.\n", "\n", "We will perform an unsupervised learning algorithm in Topic Modeling, which uses Latent Dirichlet Allocation (LDA) Model, and LDA Mallet (Machine Learning Language Toolkit) Model, on an entire department's decision making rationales.\n", "\n", "We will also determine the dominant topic associated to each rationale, as well as determining the rationales for each dominant topics in order to perform quality control analysis.\n", "\n", "Note: Although we were given permission to showcase this project, however, we will not showcase any relevant information from the actual dataset for privacy protection.\n", "
\n", "
\n", "\n", "**Benefits:**\n", "- Efficiently determine the main topics of rationale texts in a large dataset\n", "- Improve the quality control of decisions based on the topics that were extracted\n", "- Conveniently determine the topics of each rationale\n", "- Extract detailed information by determining the most relevant rationales for each topic\n", "
\n", "
\n", "\n", "**Robustness:**\n", "\n", "To ensure the model performs well, I will take the following steps:\n", "- Run the LDA Model and the LDA Mallet Model to compare the performances of each model\n", "- Run the LDA Mallet Model and optimize the number of topics in the rationales by choosing the optimal model with highest performance\n", "\n", "Note that the main different between LDA Model vs. LDA Mallet Model is that, LDA Model uses Variational Bayes method, which is faster, but less precise than LDA Mallet Model which uses Gibbs Sampling. \n", "
\n", "
\n", "\n", "**Assumption:**\n", "- We are using data with a sample size of 511, and assuming that this dataset is sufficient to capture the topics in the rationale\n", "- We're also assuming that the results in this model is applicable in the same way if we were to train an entire population of the rationale dataset with the exception of few parameter tweaks\n", "
\n", "
\n", "\n", "**Future:**\n", "\n", "This model is an innovative way to determine key topics embedded in large quantity of texts, and then apply it in a business context to improve a Bank's quality control practices for different business lines. However, since we did not fully showcase all the visualizations and outputs for privacy protection, please refer to \"[Employer Reviews using Topic Modeling](https://nbviewer.jupyter.org/github/mick-zhang/Employer-Reviews-using-Topic-Modeling/blob/master/Topic%20Employer%20Github.ipynb?flush_cache=true)\" for more detail." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 2 Data Overview" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
UnderwriterDeal NumberDecisionDeal Notes
\n", "
" ], "text/plain": [ "Empty DataFrame\n", "Columns: [Underwriter, Deal Number, Decision, Deal Notes]\n", "Index: []" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "csv = (\"audit_rating_banking.csv\")\n", "df = pd.read_csv(csv, encoding='latin1') # Solves enocding issue when importing csv\n", "df.head(0)" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "After importing the data, we see that the \"Deal Notes\" column is where the rationales are for each deal. This is the column that we are going to use for extracting topics. \n", "\n", "Note that actual data were not shown for privacy protection." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "(511, 1)" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = df[['Deal Notes']]\n", "df.shape" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "hidden": true, "scrolled": false }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Deal Notes
2Unfortunately I will have
3Were going to pass
4Credit: main applicant has
\n", "
" ], "text/plain": [ " Deal Notes\n", "2 Unfortunately I will have\n", "3 Were going to pass\n", "4 Credit: main applicant has" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df1 = df.copy()\n", "df1[\"Deal Notes\"] = df1[\"Deal Notes\"].apply(lambda x : x.rsplit(maxsplit=len(x.split())-4)[0]) # sets the character limit to 4 words\n", "df1.loc[2:4, ['Deal Notes']]" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "As a expected, we see that there are 511 items in our dataset with 1 data type (text).\n", "\n", "I have also wrote a function showcasing a sneak peak of the \"Rationale\" data (only the first 4 words are shown)." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 3 Data Cleaning" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "We will use regular expressions to clean out any unfavorable characters in our dataset, and then preview what the data looks like after the cleaning." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "hidden": true }, "outputs": [], "source": [ "data = df['Deal Notes'].values.tolist() # convert to list\n", "\n", "# Use Regex to remove all characters except letters and space\n", "import re\n", "data = [re.sub(r'[^a-zA-Z ]+', '', sent) for sent in data]\n", "\n", "# Preview the first list of the cleaned data\n", "from pprint import pprint\n", "pprint(data[:1])" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Note that output were omitted for privacy protection. However the actual output here are text that has been cleaned with only words and space characters." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 4 Pre-Processing" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "With our data now cleaned, the next step is to pre-process our data so that it can used as an input for our LDA model.\n", "\n", "We will perform the following:\n", "- Breakdown each sentences into a list of words through Tokenization by using Gensim's `simple_preprocess`\n", "- Additional cleaning by converting text into lowercase, and removing punctuations by using Gensim's `simple_preprocess` once again\n", "- Remove stopwords (words that carry no meaning such as to, the, etc) by using NLTK's `corpus.stopwords`\n", "- Apply Bigram and Trigram model for words that occurs together (ie. warrant_proceeding, there_isnt_enough) by using Gensim's `models.phrases.Phraser`\n", "- Transform words to their root words (ie. walking to walk, mice to mouse) by Lemmatizing the text using `spacy.load(en)` which is Spacy's English dictionary" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hidden": true, "scrolled": true }, "outputs": [], "source": [ "# Implement simple_preprocess for Tokenization and additional cleaning\n", "import gensim\n", "from gensim.utils import simple_preprocess \n", "def sent_to_words(sentences):\n", " for sentence in sentences:\n", " yield(gensim.utils.simple_preprocess(str(sentence), deacc=True)) \n", " # deacc=True removes punctuations \n", "data_words = list(sent_to_words(data))\n", "\n", "# Remove stopwords using gensim's simple_preprocess and NLTK's stopwords\n", "from nltk.corpus import stopwords\n", "stop_words = stopwords.words('english')\n", "stop_words.extend(['from', 'subject', 're', 'edu', 'use']) # Add additional stop words\n", "def remove_stopwords(texts):\n", " return [[word for word in simple_preprocess(str(doc)) if word not in stop_words] for doc in texts]\n", "data_words_nostops = remove_stopwords(data_words)\n", "\n", "\n", "# Create and Apply Bigrams and Trigrams\n", "bigram = gensim.models.Phrases(data_words, min_count=5, threshold=100) # higher threshold fewer phrases\n", "trigram = gensim.models.Phrases(bigram[data_words], threshold=100)\n", "bigram_mod = gensim.models.phrases.Phraser(bigram) # Faster way to get a sentence into a trigram/bigram\n", "trigram_mod = gensim.models.phrases.Phraser(trigram)\n", "def make_trigram(texts):\n", " return [trigram_mod[bigram_mod[doc]] for doc in texts]\n", "data_words_trigrams = make_trigram(data_words_nostops)\n", "\n", "\n", "# Lemmatize the data\n", "import spacy\n", "nlp = spacy.load('en', disable=['parser', 'ner'])\n", "def lemmatization(texts, allowed_postags=['NOUN', 'ADJ', 'VERB', 'ADV']):\n", " texts_out = []\n", " for sent in texts:\n", " doc = nlp(\" \".join(sent)) # Adds English dictionary from Spacy\n", " texts_out.append([token.lemma_ for token in doc if token.pos_ in allowed_postags])\n", " # lemma_ is base form and pos_ is lose part\n", " return texts_out\n", "data_lemmatized = lemmatization(data_words_trigrams, allowed_postags=['NOUN', 'ADJ', 'VERB', 'ADV'])\n", "\n", " \n", "# Preview the data \n", "print(data_lemmatized[:1])" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Note that output were omitted for privacy protection. However the actual output here are text that are Tokenized, Cleaned (stopwords removed), Lemmatized with applicable bigram and trigrams." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 5 Prepare Dictionary and Corpus" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Now that our data have been cleaned and pre-processed, here are the final steps that we need to implement before our data is ready for LDA input:\n", "- Create a dictionary from our pre-processed data using Gensim's `corpora.Dictionary`\n", "- Create a corpus by applying \"term frequency\" (word count) to our \"pre-processed data dictionary\" using Gensim's `.doc2bow`" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "hidden": true, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[(0, 1), (1, 1), (2, 1), (3, 1), (4, 1), (5, 1), (6, 1), (7, 1), (8, 1), (9, 1), (10, 1), (11, 1), (12, 1), (13, 1), (14, 1), (15, 1)]]\n" ] } ], "source": [ "import gensim.corpora as corpora\n", "id2word = corpora.Dictionary(data_lemmatized) # Create dictionary\n", "texts = data_lemmatized # Create corpus\n", "corpus = [id2word.doc2bow(text) for text in texts] # Apply Term Frequency\n", "print(corpus[:1]) # Preview the data" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "We can see that our corpus is a list of every word in an index form followed by count frequency." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "hidden": true, "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "'brother'" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "id2word[0]" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "We can also see the actual word of each index by calling the index from our pre-processed data dictionary." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hidden": true, "scrolled": false }, "outputs": [], "source": [ "[[(id2word[id], freq) for id, freq in cp] for cp in corpus[:1]]" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Lastly, we can see the list of every word in actual word (instead of index form) followed by their count frequency using a simple `for` loop.\n", "\n", "Note that output were omitted for privacy protection. However the actual output here are a list of text showing words with their corresponding count frequency.\n", "\n", "Now that we have created our dictionary and corpus, we can feed the data into our LDA Model." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 6 LDA Model" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "**Latent (hidden) Dirichlet Allocation** is a generative probabilistic model of a documents (composites) made up of words (parts). The model is based on the probability of words when selecting (sampling) topics (category), and the probability of topics when selecting a document.\n", "\n", "Essentially, we are extracting topics in documents by looking at the probability of words to determine the topics, and then the probability of topics to determine the documents. \n", "\n", "There are two LDA algorithms. The **Variational Bayes** is used by Gensim's **LDA Model**, while **Gibb's Sampling** is used by **LDA Mallet Model** using Gensim's Wrapper package.\n", "\n", "Here is the general overview of Variational Bayes and Gibbs Sampling:\n", "- **Variational Bayes**\n", " - Sampling the variations between, and within each word (part or variable) to determine which topic it belongs to (but some variations cannot be explained)\n", " - Fast but less accurate\n", "- **Gibb's Sampling (Markov Chain Monte Carlos)**\n", " - Sampling one variable at a time, conditional upon all other variables\n", " - Slow but more accurate" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hidden": true }, "outputs": [], "source": [ "# Build LDA Model\n", "lda_model = gensim.models.ldamodel.LdaModel(corpus=corpus, id2word=id2word, num_topics = 9, random_state = 100,\n", " update_every = 1, chunksize = 100, passes = 10, alpha = 'auto',\n", " per_word_topics=True) # Here we selected 9 topics\n", "pprint(lda_model.print_topics())\n", "doc_lda = lda_model[corpus]" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "After building the LDA Model using Gensim, we display the 10 topics in our document along with the top 10 keywords and their corresponding weights that makes up each topic.\n", "\n", "Note that output were omitted for privacy protection.. However the actual output is a list of the 9 topics, and each topic shows the top 10 keywords and their corresponding weights that makes up the topic." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 7 LDA Model Performance" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "hidden": true, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Perplexity: -6.876382108603263\n", "\n", "Coherence Score: 0.4136469957174781\n" ] } ], "source": [ "# Compute perplexity\n", "print('Perplexity: ', lda_model.log_perplexity(corpus))\n", "\n", "# Compute coherence score\n", "from gensim.models import CoherenceModel\n", "coherence_model_lda = CoherenceModel(model=lda_model, texts=data_lemmatized, dictionary=id2word, coherence='c_v')\n", "coherence_lda = coherence_model_lda.get_coherence()\n", "print('Coherence Score: ', coherence_lda)" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "In order to determine the accuracy of the topics that we used, we will compute the Perplexity Score and the Coherence Score. The Perplexity score measures how well the LDA Model predicts the sample (the lower the perplexity score, the better the model predicts). The Coherence score measures the quality of the topics that were learned (the higher the coherence score, the higher the quality of the learned topics).\n", "\n", "Here we see a **Perplexity score of -6.87** (negative due to log space), and **Coherence score of 0.41**. \n", "\n", "Note: We will use the Coherence score moving forward, since we want to optimizing the number of topics in our documents." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true, "hidden": true }, "source": [ "## 7.1 Visualize LDA Model" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hidden": true }, "outputs": [], "source": [ "import warnings\n", "warnings.filterwarnings(\"ignore\", category=FutureWarning) # Hides all future warnings\n", "import pyLDAvis\n", "import pyLDAvis.gensim \n", "pyLDAvis.enable_notebook()\n", "vis = pyLDAvis.gensim.prepare(lda_model, corpus, id2word)\n", "vis" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "We are using pyLDAvis to visualize our topics. \n", "\n", "For interpretation of pyLDAvis:\n", "- Each bubble represents a topic\n", "- The larger the bubble, the more prevalent the topic will be\n", "- A good topic model has fairly big, non-overlapping bubbles scattered through the chart (instead of being clustered in one quadrant)\n", "- Red highlight: Salient keywords that form the topics (most notable keywords)\n", "\n", "Note that output were omitted for privacy protection." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 8 LDA Mallet Model" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Now that we have completed our Topic Modeling using \"Variational Bayes\" algorithm from Gensim's LDA, we will now explore Mallet's LDA (which is more accurate but slower) using Gibb's Sampling (Markov Chain Monte Carlos) under Gensim's Wrapper package.\n", "\n", "Mallet's LDA Model is more accurate, since it utilizes Gibb's Sampling by sampling one variable at a time conditional upon all other variables." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "hidden": true }, "outputs": [], "source": [ "import os\n", "from gensim.models.wrappers import LdaMallet\n", "os.environ.update({'MALLET_HOME':r'/Users/Mick/Desktop/mallet/'}) # Set environment\n", "mallet_path = '/Users/Mick/Desktop/mallet/bin/mallet' # Update this path\n", "\n", "# Build the LDA Mallet Model\n", "ldamallet = LdaMallet(mallet_path,corpus=corpus,num_topics=9,id2word=id2word) # Here we selected 9 topics again\n", "pprint(ldamallet.show_topics(formatted=False))" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "After building the LDA Mallet Model using Gensim's Wrapper package, here we see our 9 new topics in the document along with the top 10 keywords and their corresponding weights that makes up each topic.\n", "\n", "Note that output were omitted for privacy protection. However the actual output is a list of the 9 topics, and each topic shows the top 10 keywords and their corresponding weights that makes up the topic." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true, "hidden": true }, "source": [ "## 8.1 LDA Mallet Model Performance" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "hidden": true, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Coherence Score: 0.4102038587308669\n" ] } ], "source": [ "# Compute coherence score\n", "coherence_model_ldamallet = CoherenceModel(model=ldamallet, texts=data_lemmatized, dictionary=id2word, coherence=\"c_v\")\n", "coherence_ldamallet = coherence_model_ldamallet.get_coherence()\n", "print('\\nCoherence Score: ', coherence_ldamallet)" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Here we see the Coherence Score for our **LDA Mallet Model** is showing **0.41** which is similar to the LDA Model above. Also, given that we are now using a more accurate model from **Gibb's Sampling**, and combined with the purpose of the Coherence Score was to measure the quality of the topics that were learned, then our next step is to improve the actual Coherence Score, which will ultimately improve the overall quality of the topics learned.\n", "\n", "To improve the quality of the topics learned, we need to find the optimal number of topics in our document, and once we find the optimal number of topics in our document, then our Coherence Score will be optimized, since all the topics in the document are extracted accordingly without redundancy." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 9 Finding the Optimal Number of Topics for LDA Mallet Model" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "We will use the following function to run our **LDA Mallet Model**:\n", "\n", " compute_coherence_values\n", " \n", "Note: We will trained our model to find topics between the range of 2 to 12 topics with an interval of 1." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "hidden": true, "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd4lFX2wPHvSYMAoSYB0gAhgPSSIHaMqNgAK1hZVxdXxbKurq7u+lPXLbouuqtY0NUVG6CrgIJiBRstSOgGQk+ABILUmH5+f8zEHWNIBsibdzJzPs+Th3nvvOVkmMyZe+977xVVxRhjjKlNmNsBGGOMCXyWLIwxxtTJkoUxxpg6WbIwxhhTJ0sWxhhj6mTJwhhjTJ0sWRhjjKmTJQtjjDF1cjRZiMgIEckWkRwRubeW/S4RERWRtGrlKSJyUETucjJOY4wxtYtw6sQiEg5MAs4CcoElIjJLVddU2y8GuB1YVMNpJgIf+HO92NhY7dy58zHFbIwxoWbp0qW7VTWurv0cSxbAECBHVTcCiMhUYBSwptp+fwIeBe72LRSR0cAm4JA/F+vcuTOZmZnHGrMxxoQUEdniz35ONkMlAtt8tnO9ZT8SkUFAsqrOrlbeArgHeMjB+IwxxvjJtQ5uEQnD08z02xqefhB4QlUP1nGO8SKSKSKZu3btciBKY4wx4GwzVB6Q7LOd5C2rEgP0AeaJCEAHYJaIjAROAC4VkceA1kCliBSr6tO+F1DVycBkgLS0NJs+1xhjHOJkslgCpIpIFzxJYixwZdWTqroPiK3aFpF5wF2qmgmc6lP+IHCweqLwR1lZGbm5uRQXFx/t7+C4pk2bkpSURGRkpNuhGGPMYTmWLFS1XEQmAHOBcOAlVV0tIg8Dmao6y6lrV8nNzSUmJobOnTvjrb0EFFWlsLCQ3NxcunTp4nY4xhhzWE7WLFDVOcCcamUPHGbfYYcpf/Bor19cXBywiQJARGjXrh3W32KMCXRBP4I7UBNFlUCPzxhjIASShTHGOGFV3j6+WBc6rQKWLIwx5giVlFdw46tLuem1pRwqKXc7nAZhycIYY47QtCXbyNv7A4dKK5i9Yofb4TQISxYNYMqUKfTr14/+/ftzzTXXuB2OMeYY/FBawVOf5ZDeuQ3d4lswdclWt0NqEI7eDRVIHnpvNWu276/Xc/ZKaMn/Xdi71n1Wr17NI488wjfffENsbCx79uyp1xiMMQ1ryoLN7DpQwqQrB7Eidy+PzF7LuvwDdG8f43ZojrKahcM+++wzLrvsMmJjPeMP27Zt63JExpijtb+4jGfnb+D07nEM6dKWiwYmEhkuTF28re6DG7mQqVnUVQMwxpi6/PvLTewtKuOus3sA0K5FE87u3YF3luVyz7k9aBIR7nKEzrGahcMyMjJ46623KCwsBLBmKGMaqT2HSnnxy42c26cDfZNa/Vg+Nj2ZvUVlfLQ638XonGfJwmG9e/fm/vvv5/TTT6d///7ceeedbodkjDkKz83fQFFZBXee1f0n5Sd3jSWxdTTTlgR3U1TINEO5ady4cYwbN87tMIwxRyl/fzGvfLOZiwYmklqtIzssTBiTnszEj9exbU8RyW2buRSls6xmYYwxdXjqs/VUVCp3nNm9xucvHZxEmBDUtQtLFsYYU4uthUVMXbyNsUOSSWlXc60hoXU0p3eP462l2yivqGzgCBtG0CcL1cBeEynQ4zMm1D356TrCw4RbM1Jr3W/skBTy95cwP0jniwrqZNG0aVMKCwsD9gO5aj2Lpk2buh2KMaYG6/MP8O6yPK49sRPtW9b+d5rRM57YFk2YGqRNUUHdwZ2UlERubm5ArxdRtVKeMSbwTPx4Hc0iw7lpWLc6940MD+PSwUm88OVGCvYXE19HcmlsgjpZREZG2gp0xpijsjJ3Hx+s2sltZ6bStnmUX8eMSU/mufkbeGtpLrecUXeCaUyCuhnKmGNVXFYRsM2YxlmPf5RN62aR3HCq/184u8Q254QubZmeuY3KyuB631iyMOYwthYWcfLfPuOh99a4HYppYIs37WH+ul38+vSutGwaeUTHXjEkhS2FRSzcVOhQdO6wZGFMDQ6VlDP+1UwKD5Xy2sIt5H5f5HZIpoGoKo/PzSYupgnjTux8xMeP6NOBlk0jgm7MhSULY6pRVe5+eznr8g/w2CX9CBPhmXkb3A7LNJAv1u9m8eY93JrRjeioI58YsGlkOBcNTOSDVTvZW1TqQITucDRZiMgIEckWkRwRubeW/S4RERWRNO/2WSKyVERWev/NcDJOY3w9O38Dc1bu5J4RPbk8PZkx6cm8lbnNahchoKpWkdQmmrHpKUd9njHpKZSWVzJjWV49Rucux5KFiIQDk4BzgV7AFSLSq4b9YoDbgUU+xbuBC1W1LzAOeNWpOI3x9Xl2AX+fm82F/RMYf9pxANw0rCuC1S5CwdzVO1mZt4/bz0wlKuLoPx57JbSkX1Irpi7ZFjQ3SDhZsxgC5KjqRlUtBaYCo2rY70/Ao0BxVYGqLlPV7d7N1UC0iDRxMFZj2Lz7ELe/uYyeHVry2CX9EBHAM5XD5elJvJXpWXfZBKeKSuXxj9bRNa45Fw1MPObzjUlP5rudB1ieu68eonOfk8kiEfDt4cn1lv1IRAYByao6u5bzXAJ8q6ol1Z8QkfEikikimYE88M4EvoPeDu3wMGHyNYN/1lZ9s3dQ1jOf57gRnmkAM7PyyCk4yJ1n9SAi/Ng/Gkf2TyA6MpxpQbJGt2sd3CISBkwEflvLPr3x1DpurOl5VZ2sqmmqmhYXF+dMoCboqSp3TV9OTsFBnr5yUI1TTCe0jmZMejLTrXYRlErLK3nik3X0TmjJuX061Ms5Y5pGckG/jszK2s6hkvJ6OaebnEwWeUCyz3aSt6xKDNAHmCcim4GhwCyfTu4k4F3gWlW1xmLjmEmf5/Dh6p3cd97xnNwt9rD73WS1i6A1PXMb2/b8wF1n9yAsTOrtvGOHJHOotILZK3bU2znd4mSyWAKkikgXEYkCxgKzqp5U1X2qGquqnVW1M7AQGKmqmSLSGpgN3KuqXzsYowlxn32Xzz8+XsfoAQlcf0rtI3UTW0dzeZqndrHdahdBo7isgqc+W09apzYM61G/LRSDUtrQLb4FbwZBU5RjyUJVy4EJwFxgLTBdVVeLyMMiMrKOwycA3YAHRCTL+xPvVKwmNG3YdZDb38yid0JL/ubToV2bm73z/Twzz2oXweLVBVvI31/CXef08Os9cCREhLHpySzbupfsnQfq9dwNzdE+C1Wdo6rdVbWrqv7ZW/aAqs6qYd9hqprpffyIqjZX1QE+PwVOxmpCy4HiMsZPySQyIoznr0mjaaR/g6+qahfTlljtIhgcKC7jmXk5nJoay9Dj2jlyjYsHJREZLo1+RLeN4DYhp7JSuXP6cjYXFjHpykEkto4+ouOtdhE8XvpqM98XlXHX2T0cu0bb5lGc3bsD7yzLpaS8wrHrOM2ShQk5//psPR+vyecP5x/PiV2P/NtkYutoLktLZvqSXKtdNGLfHyrlxS83ck7v9vRPbu3otcamJ7O3qIyPVuc7eh0nWbIwIeXjNfk8+cl6Lh6UyC9O6nzU57l5WFcU5Vkb1d1oPffFBg6WlvNbB2sVVU7uGkti62imNuKObksWJmTkFBzkN9Oy6JfUir9c1PeYOjOT2jTj0sGevosd+6x20dgU7C/mlW82M3pAIt3bxzh+vbAwYUx6Ml/nFLK1sHHOMWbJwoSE/d4O7aaRYTx39WC/O7Rrc8sZXalUq100Rk9/nkN5hXLH8NQGu+ZlaUmEiWdMR2NkycIEvcpK5TdTs9i6x9OhnXCEHdqHk9SmGZelJTN1sdUuGpNte4p4c/FWLk9PplO75g123Y6tohnWI563lm6jvKKywa5bXyxZmKD35Cfr+PS7Ah64sBcn1PPtkVa7aHz++el6RIRbMxp+jewx6cnk7y9h/rrGN5edJQsT1D5ctYN/fZbD5WlJXDO0U72f31O7SLLaRSORU3CQd77N5dqhnejYqn5qmEcio2c8sS2a8ObixtcUZcnCBK11+Qf47fTl9E9uzcOj+tT76NwqNw/rRqUqz1ntIuA98ck6oiPDuWlYV1euHxkexqWDk/g8u4D8/cV1HxBALFmYoLSvyNOhHR0VwfP11KF9OMltPbWLNxdvY+e+xvUBEEpW5e1j9oodXH9KF9q1cG95nDHpyVRUKm8vzXUthqNhycIEnYpK5fZpy8jb+wPPXT2IDq2aOn7NqtrFszaqO2BN/HgdraIjucG7AqJbusQ2Z+hxbZmeuY3Kysazip4lCxN0/vFRNvOyd/HgyN6kdW7bINdMbtuMSwcn8eYSq10EoqVb9vDZdwXcePpxtGwa6XY4jE1PYUthEQs3Fbodit8sWZigMnvFDp6Zt4ErhiRz1Qn136Fdm1vO6EZlpfLcfOu7CCSqymMfZhPboskxjdqvTyP6dKBl04hGNbmgJQsTNL7buZ+73lrOoJTWPDiyd4NfP7ltMy4ZlMQbi7c2us7LYPZVzm4WbdrDhDO60iwqwu1wAGgaGc5FAxP5YNVO9haVuh2OXyxZmKCwt6iU8VOWEtM0gueuHkyTCOc6tGtTVbuwcReBQVV5fG42ia2jueKEFLfD+Ykx6SmUllfy7rK8uncOAJYsTKNXUanc+uYydu4r5rlrBhPf0vkO7cNJaWe1i0Dy0Zp8lufu4/YzU137AnE4vRJa0i+pFVMXb0M18Du6LVmYRu+xud/x5frdPDyqN4NS2rgdjtUuAkRFpTLxo3UcF9uciwcluh1Ojcamp5Cdf4DlufvcDqVOlixMo/be8u08P38jVw9NYeyQwGhmSGnXjIsHJVrtwmXvLd9Odv4BfnNWdyLCA/Oj7sL+HYmODGdaI5i6PDBfQWP8sGb7fu5+eznpndvwwAUN36FdmwlnpFJhd0a5pqyikic+WcfxHVtyft+ObodzWDFNI7mgX0dmZW3nUEm52+HUypKFaZS+P1TK+FczaR0dxaSrBhEVEVhvZU/fRSJvLNpKgdUuGtzbS3PZUljEXWd3JyzMmWle6svYIckcKq3g/RXb3Q6lVo7+hYnICBHJFpEcEbm3lv0uEREVkTSfst97j8sWkXOcjNM0LuUVlUx481sKDpR4OrRj3OvQrs2EM1Ipr1SetdpFgyouq+Bfn65nUEprMnrGux1OnQaltKFbfAumBviYC8eShYiEA5OAc4FewBUi0quG/WKA24FFPmW9gLFAb2AE8Iz3fMbw6Iff8XVOIY+M7sMAh9dOPhYp7Zpx8UCrXTS01xdtZce+Yu46p4djk0fWJxFhbHoyy7buJXvnAbfDOSwnaxZDgBxV3aiqpcBUYFQN+/0JeBTw/WsaBUxV1RJV3QTkeM9nQtzMrDxe+HIT407sxOVpyW6HU6cJGd0or1Sem7/R7VCO2fx1uzj3n1/yj4+yA3Y69kMl5TzzeQ6ndIvlpK6xbofjt4sHJREZLgE9otvJZJEI+P7mud6yH4nIICBZVWcf6bEm9KzK28fv3l7BkC5t+cMFP6ukBqRO7Zpz8cBEXl+0pVHXLj7/roBfvZLJrgMlPP15Dqc8+jk3v76UhRsLA2qMwMtfb6LwUCl3ndPD7VCOSNvmUZzduwPvLMuluKzC7XBq5FqvoIiEAROB3x7DOcaLSKaIZO7a1fhWnjL+KzxYwo2vLqVd8yieuWoQkQF6K2RNGnvt4rPv8rnx1aV079CCT+48jS/uPoMbTunC1zmFjJ28kBFPfsnri7a4fjfPvqIynv9iI2f1ah/QzZOHMzY9mb1FZXy0Jt/tUGrk5F9cHuDbTpDkLasSA/QB5onIZmAoMMvbyV3XsQCo6mRVTVPVtLi4uHoO3wSKsopKJryxjN0HS3j+mjRiXVyL4Gh0ateci6pqFwcaV+3ikzWeRNGjQwyvXz+U1s2iSG7bjN+fdzwLf38mj13Sj4hw4f53VzH0L5/y0Hur2bjroCuxPv/FBg6WlPPbs7u7cv1jdXLXWBJbRwfsmAsnk8USIFVEuohIFJ4O61lVT6rqPlWNVdXOqtoZWAiMVNVM735jRaSJiHQBUoHFDsZqAthf5qxlwcZC/npxX/omtXI7nKMy4QxP7eL5RlS7+HhNPje9vpReHVvy2g0n0KrZT6f2jo4K5/L0ZN6/9RT+e9NJZBwfz2sLt5Dxj/lc+9JiPl2bT0UDrddQcKCYl7/ezMj+CfTs0LJBrlnfwsKEMenJfJ1TyNbCIrfD+RnHkoWqlgMTgLnAWmC6qq4WkYdFZGQdx64GpgNrgA+BW1Q1MBvyjKP+uzSXl7/ezC9P7sLFg5LcDueodY5tzugBiby2sHHULuau3snNry+lV0Irplx/Aq2iD78GhIgwuFMb/jl2IF/fm8GdZ3Une+d+rn8lk2GPf87kLzY4PrPqM59voLSikt8Mb5y1iiqXpSURJjA9M/A6uiWQOqeORVpammZmZrodhqlHK3L3culzCxic0oZXrx8SsFM2+Gvz7kOcOXE+153UOaA76D9ctYMJbyyjb1IrXvnlkKNaLKisopKPVuczZcFmFm3aQ5OIMEYNSODaEzvTJ7F+a4d5e3/gjL/P45LBifz14n71em43/PI/S1i9fR9f35PRIO95EVmqqml17de4//pM0Np1wNOhHdeiCU9fObDRJwrwqV0EcN/FnJU7uOWNZfRLasWUo0wUAJHhYZzfryPTbjyRD+84lUsGJ/He8h1c8NRXXPLsN8zMyqO0vLJeYv7XJ+sBuDUjtV7O57Yx6cnk7y9hXnZg3bTj11+giESLSOO6F800OpWVytIt3/On99dw4VNf8X1RKc9fM5h2jaxDuza3ZnSjrEKZHIB9F7NX7ODWN5cxMLk1U64/gZh6Wn60Z4eW/OWiviy870z+eEEv9hwq5fapWZz0t8+Y+PG6Y1qGduOug7z9bS5XD+1EQuvoeonXbRk944lt0STgRnTXuWyUiFwIPA5EAV1EZADwsKrW2u9gjD8qK5WlW79n9oodfLhqJzv3FxMVHsZp3WO5/pTj6r3Jwm2dY5szakACry3awo2ndyUuJjAS4XvLt3PHtCwGpbTm5euG0KJJ/a8o1yo6kutP6cJ1J3Xmy5zdTPlmM099tp5Jn+cwoncHrj2xE0O6tD2iUddPfLKeJhFh3HxG13qP1y2R4WFcOjiJF77cSP7+Ytq7uD6LL3/eEQ/iGT09D0BVs7x3KBlzVCoqlSWb9/DByh18sGonBQdKiIoIY1j3OH7frycZPePr7VttILo1I5UZy/KY/MUG7j/f/b6LmVl5/GZaFmmd2vLydek0dyBR+AoLE07vHsfp3ePYWljEa4u2MG3JNmav3EHPDjFce2JnRg9MqHMJ1DXb9/Pe8u1MOKNbo7udui5j0pN5bv4G3l6ayy1ndHM7HMC/ZFGmqvuqZfvg6BU3Daa8opLFm/cwZ+UOPlyVz+6DJTSJCOOMHvGc168jGT3jHfk2G4i6xDZn9MBEXl24hfGnuVu7qEoU6Z3b8tIvnE8U1aW0a8Z95x3Pb4Z3Z9byPF75Zgv3vbuSv36wlsvTkrlmaCc6xzav8diJH2fTsmkEvzrtuAaNuSF0iW3O0OPaMj1zGzed3jUgZs71552xWkSuBMJFJBW4DfjG2bBMMCivqGThxj3MWbWDuat2UniolOjIcDJ6xnNu3w6c0SO+wT+cAkUg1C7eXZbLb6cvZ0gXT6Ko65u8k6KjwhmTnsLlacl8u/V7XvlmC698s5l/f7WJ07vHMe6kTgzrHv/jh+a3W7/nk7UF3H1Oj1pv623MxqancMe0LBZuLOSkbu7Pc1XnrbMi0gy4HzjbWzQXeERVA+p2Drt1NjCUVVSyYEMhc1buYO7qnXxfVEazqHDOPL495/XpwLAe8URH2QTCAHdOy2LOqh18dU9GgzejvL00l7vfXs6Jx7Xj3+PSA/L/pGB/MW8u3uYd+V5CSttmXDO0E5elJXHz69+yLv8AX/zuDFeTnJOKyyoY8udPGNYjnn9dMdCx6/h762ytycI7LfijqnpXfQbnBEsW7iktr+TrDbv5YOUOPlqTz96iMlo0ieDM4+M5r29HTu8eR9PIwPswctvGXQcZPnE+N5x6HPedd3yDXfetzG387r8rOKlrO168NjATha+yikrmrt7JlAVbWLxpD1ERYZSWV/LABb345SnB3X36fzNX8ebibSy670zaNI9y5Br+JotaU7KqVojIKfUXlgkWJeUVfLV+N3NW7uTjNTvZX1xOTJMIzurVnnP7duTU1FhLEHU4Lq4FowYkMmXBZsafdlyD1C6mL9nGPe+s4JRusbxwbVqj+D+KDA/jgn4JXNAvgbU79jNlwWZ27CvmyhMCY811J41JT+GVBVuYkZXHdSe7mxj9qb8tE5FZwFvAoapCVX3HsahMQCouq+DL9buZs3IHn6zJ50BJOS2bRnB27w6c17cDJ3eLpUlE4H/4BJIJGd08a3R8sZHfO1y7mLp4K/e+s5LTuscx+ZrBjSJRVHd8x5ZBMUrbX70SWtI/qRVTF2/jFyd1dnUxJ3+SRVOgEMjwKVPAkkUIKC6rYF72Lj5YtYNP1xZwsKSc1s0iObdvB87r25GTusYG3PrXjUnXH2sXW/iVg7WLNxZt5b53V3J69zieb6SJIlSNSU/hvndXkrVtLwNT2rgWR53JQlWva4hATOD4obSCedkFzF65g8++K6CotII2zSK5sH9Hzu3TkRO7tmtU60kEOqdrF68v2sL9767ijB5xPHu1JYrG5sL+HfnT+2uYtmRbYCcLEUkCngJO9hZ9CdyuqrlOBmbc8ey8Dfzr0/X8UFZBu+ZRjB6YyPl9O3JCl7ZBMT9TIOoa14KR/RMcqV28unALf5yxioye8Tx79SBrJmyEYppGckG/jsxavp0/XNDLtfFI/vz1v4xnfYkE78973jITRFSVx+dm8+iH33Fqaixv/OoEFt8/nL9c1JeTu8VaonDYhIxUSsoreOHL+pszasqCzfxxxiqGH2+JorEbOySZotIKZq/Y7loM/nwCxKnqy6pa7v35D2DL0gURVeWvH3zH05/ncMWQZJ67ejAndY0lPABGjYaKbvHe2sU3Wyg8WHLM53v56008MHM1Z/VqzzNXDbZE0cgNSmlDanwLVycX9CdZFIrI1SIS7v25Gk+HtwkCqspD761h8hcbufbETvx5dN+AmFogFE3ISKW4vILJx1i7+PdXm3jovTWc3as9k64cZDcgBAERzyp6y7buJXvnAVdi8Odd9EvgcmAnsAO4FLBO7yBQWanc9+4q/vPNZm44pQsPjexticJFVbWLVxccfe3ixS838qf31zCidwcmXWWJIphcPCiJyHBhqktrdNf5TlLVLao6UlXjVDVeVUeramCuKG78VlGp/O6/K3hz8VZuHtaV+88/3tV7uI3HrRmp/FBWwQtfbjriY1/4YiOPzF7LeX078NSVA+2OtSDTtnkUZ/fuwLvL8igua/hVput8N4nIKyLS2me7jYi85GxYxknlFZXcOT2Lt5fm8pvh3bn7nB6WKALEj30XCzYfUe3i+fkb+POctZzfryP/HGuJIliNTU9mb1EZH63Jb/Br+/OO6qeqe6s2VPV7wLlZrYyjyioquW3qMmZmbed3I3pw+/BUSxQB5taMbkdUu3h23gb++sF3XNg/gX+OGWCJIoid3DWWpDbRTHOhKcqfd1WYiPw4EkRE2uLfyG8TYErKK7jptW+Zs3Infzj/eG4eFhiLqpif6hYfw4X9PLWLPYdKa9130uc5PPrhd4zsn8ATl/e3W5yDXFiYMCYtma9zCtlSeKjuA+rz2n7s8w9ggYj8SUQewbOWxWP+nFxERohItojkiMi9NTz/axFZKSJZIvKViPTylkd6m79WishaEfn9kfxS5ueKyyq48dWlfLI2n4dH9eaGU4NvwZhgctuZVbWLw98Z9dSn6/n73GxGD0hgoiWKkHFpWhJhAtMzG/Y2Wn86uKcAFwP5eO6IulhVX63rOO/05pOAc4FewBVVycDHG6raV1UH4ElAE73llwFNVLUvMBi4UUQ6+/UbmZ/5obSCG17JZP66Xfzt4r5ce2Jnt0MydegWH8MF/RJ45Zuaaxf//GQ9//h4HRcPTOQflw+wRBFCOraKZliPeN7KzKW8orLBrutPB3dXYIOqPg2sAob7dnjXYgiQo6obVbUUmAqM8t1BVff7bDbnf8u1KtBcRCKAaKAU8N3X+OlQSTm/eHkx32zYzeOX9mfskOCf1jlY3Obtu3ixWu3iiY/X8cQn67h4UCJ/v6y/DZ4MQWPSkyk4UMK87F0Ndk1/vo78F6gQkW7A80Ay8IYfxyUCvvWkXG/ZT4jILSKyAU/N4jZv8dt4pkPfAWwFHlfVPTUcO15EMkUkc9euhnvRGov9xWVc+9JiMrd8z5NjB3LJ4CS3QzJHILX9T2sXqsrEj9fxz0/Xc+ngJP5+qSWKUJXRM57YFk0adES3P8miUlXL8TRFPa2qdwMd6ysAVZ2kql2Be4A/eIuHABV45qLqAvxWRH7WyK6qk1U1TVXT4uJsBhJf+4rKuObFRSzftpenrxjIyP4JbodkjsJtGd0o8vZdTPx4Hf/6dD2XpyXx2CX9LFGEsMjwMC5LS+Lz7ALy9zfMCtf+JIsyEbkCuBZ431vmzwrpeXhqIVWSvGWHMxUY7X18JfChqpapagHwNVDnsn/GY8+hUq54YSFrdxzguasHc27fesvtpoGlto/h/L4deX7+Bp76LIex6cn87eJ+NtLecHlaMhWVyttLG2YCcH+SxXXAicCfVXWTiHQB6uzgBpYAqSLSRUSigLF4Zq/9kYik+myeD6z3Pt6Kd7ElEWkODAW+8+OaIW/XgRKumLyQDbsO8sK4NIb3au92SOYY3X5mKpHhYVx5Qgp/ucjm7jIeXWKbM/S4tkxbso3KSq37gGPkz+JHa/hfXwKqugl41I/jykVkAjAXCAdeUtXVIvIwkKmqs4AJIjIcKAO+B8Z5D58EvCwiqwEBXlbVFUf2q4We/P3FXPnCQrbvLeblX6RzUrdYt0My9SC1fQzf/vEsmru0joEJXGPTU7hjWhYLNxY6/vfu6LtPVec0U9ABAAAWZUlEQVQAc6qVPeDz+PbDHHcQz+2zxk/b9/7AlS8sZNeBEl755RCGdGnrdkimHlmiMDUZ0acDLWdGMHXJtsadLEzD2LaniCteWMi+H8p49YYTGOTi0ovGmIbTNDKch0f1IbltM8ev5XeyEJFmqlrkZDDmyG3efYgrX1jIodIK3rhhKH2TWrkdkjGmAY0e+LMRCY7wZ1DeSSKyBm8Hs4j0F5FnHI/M1Cmn4CCXP7+A4vJK3vyVJQpjjHP8uRvqCeAcvKvjqepy4DQngzJ1y955gLGTF1CpMHX8UHoltHQ7JGNMEPNrQhlVrT5MsOFX3jA/WpW3j7GTFxAeJky7cSjd28e4HZIxJsj502exTUROAlREIoHbgbXOhmUOZ/m2vVzz70XENI3kjV+dQKd2zd0OyRgTAvypWfwauAXPvE55wADvtmlgS7fs4eoXF9GqWSTTbhxqicIY02D8GZS3G7iqAWIxtVi0sZDr/rOE9i2b8savTqBjq2i3QzLGhBBbg7sR+DpnN+NeXkxC62imjR9qicIY0+BsDe4A93l2Adf9Zwmd2zVn6vihxLds6nZIxpgQZGtwB7CP1+Rz45SldG/fgjd/NZTYFk3cDskYE6L8+dCvWoP7LTyT+l0K/NnRqAxzVu7gtjeX0TuxFVN+OYRW0f7MCm+MMc7wp4N7iogsBc7wFl3snYnWOGRmVh53Tl/OwOTWvHxdOjFNLVEYY9zlb3PSd3imEI8AEJEUVd3qWFQh7O2ludz99nJO6NKWf49Lt9lGjTEBoc5PIhG5Ffg/IB/PyG0BFOjnbGih541FW7l/xkpO6RbL5GvSiI4KdzskY4wB/KtZ3A70UNVCp4MJZfOyC7jv3ZVk9IznmasG0TTSEoUxJnD4Nd0HsM/pQELd64u20r5lE569ehBNIixRGGMCiz/JYiMwT0RmAyVVhao60bGoQszeolLmZRfwi5M6W6IwxgQkf5LFVu9PlPfH1LM5K3dSVqGMGtAwi5gYY8yR8ufW2YfAVspz0oysPLrFt6C3rUlhjAlQ/swNdeLRrpQnIiNEJFtEckTk3hqe/7WIrBSRLBH5SkR6+TzXT0QWiMhq7z5BOc9F7vdFLN60h9EDEhARt8Mxxpga+TPdx5McxUp5IhIOTALOBXoBV/gmA683VLWvqg4AHgMmeo+NAF4Dfq2qvYFhQJk/v1BjM2v5dgBrgjLGBDQnV8obAuSo6kZVLQWmAqOqnXe/z2ZzPOM3AM4GVngTE6paqKpBuTrfzGXbGdypDcltm7kdijHGHJY/yeInK+WJyF34t1JeIp7bbqvkest+QkRuEZENeGoWt3mLu3uvN1dEvhWR3/lxvUZn7Y79ZOcfYPSABLdDMcaYWrm+Up6qTlLVrsA9wB+8xRHAKXgWXToFuEhEzqx+rIiMF5FMEcnctWtXfYXUYGZk5RERJpzfz5KFMSaw1ZosvP0O16jqVaraXlXjVfVqP0dz5wHJPttJ3rLDmQqM9j7OBb5Q1d3eO7DmAIOqH6Cqk1U1TVXT4uLi/AgpcFRWKu9lbee07nG0bW53JBtjAlutycLbT3DlUZ57CZAqIl1EJAoYC8zy3UFEUn02zwfWex/PBfqKSDNvZ/fpQFDNdLt48x627ytmlDVBGWMaAX8G5X0lIk8D04BDVYWq+m1tB6lquYhMwPPBHw68pKqrReRhIFNVZwETRGQ4njudvgfGeY/9XkQm4kk4CsxR1dlH/usFrplZeTSLCuesXu3dDsUYY+rkT7IY4P33YZ8yBTLqOlBV5+BpQvIte8Dn8e21HPsanttng05JeQWzV+zgnN4daBZlU5AbYwKfPyO4z6hrH3Nk5mXvYn9xuTVBGWMaDX9GcLcXkX+LyAfe7V4icr3zoQWvmVl5xLaI4pRusW6HYowxfvHn1tn/4Ol3qPoavA64w6mAgt3+4jI+WVvABf0SiAj3a0ykMca4zp9Pq1hVnQ5UgqfjGv9GcJsafLhqJ6XlldYEZYxpVPxJFodEpB3eqThEZCi2GNJRm5mVR6d2zRiQ3NrtUIwxxm/+3IpzJ57xEV1F5GsgDrjU0aiCVP7+Yr7ZUMitGak2w6wxplHx526ob0XkdKAHIEC2qgblDLBOe2/5dlSxuaCMMY2Ovzf5DwE6e/cfJCKo6hTHogpSM7Ly6JfUiuPiWrgdijHGHJE6k4WIvAp0BbL4X8e2ApYsjkBOwUFW5e3njxdUX9LDGGMCnz81izSgl6pqnXuaw5qZlUeYwIX9O7odijHGHDF/7oZaBXRwOpBgpqrMzNrOyd1iiY8JytVhjTFB7rA1CxF5D09zUwywRkQWAyVVz6vqSOfDCw7fbt3L1j1F3HZmat07G2NMAKqtGerxBosiyM3MyqNJRBjn9LYZZo0xjdNhk4Wqzq96LCLtgXTv5mJVLXA6sGBRVlHJ+yt2MLxXe2KaRrodjjHGHBV/JhK8HFgMXAZcDiwSERuU56ev1u9mz6FSRg/42fLjxhjTaPhzN9T9QHpVbUJE4oBPgLedDCxYzMjKo3WzSE7v3riWfTXGGF/+3A0VVq3ZqdDP40LeoZJyPlqdz3l9OxIVYS+ZMabx8qdm8aGIzAXe9G6PAT5wLqTg8fGafH4oq7AmKGNMo+fP3FB3i8jFwCneosmq+q6zYQWHGVl5JLaOJq1TG7dDMcaYY1LbOItuQHtV/VpV3wHe8ZafIiJdVXVDQwXZGO0+WMKX63cz/rTjCAuzGWaNMY1bbQ3pTwL7ayjf533O1GL2ih1UVKo1QRljgkJtyaK9qq6sXugt6+zPyUVkhIhki0iOiNxbw/O/FpGVIpIlIl+JSK9qz6eIyEERucuf6wWSGVl59OwQQ48OMW6HYowxx6y2ZFHbUm7RdZ1YRMKBScC5QC/giurJAHhDVfuq6gDgMWBitecn0gg707cUHmLZ1r2MHmi1CmNMcKgtWWSKyK+qF4rIDcBSP849BMhR1Y2qWgpMBUb57qCqvs1czfEu3eq9zmhgE7Daj2sFlJlZ2xGBkf1tkSNjTHCo7W6oO4B3ReQq/pcc0oAo4CI/zp0IbPPZzgVOqL6TiNyCZ+nWKCDDW9YCuAc4CzhsE5SIjAfGA6SkpPgRkvNUlRlZeQzp3JaE1nVWwIwxplE4bM1CVfNV9STgIWCz9+chVT1RVXfWVwCqOklVu+JJDn/wFj8IPKGqB+s4drKqpqlqWlxcYIyQXpW3n427DlkTlDEmqPgzzuJz4POjOHcekOyzneQtO5ypwLPexycAl4rIY3j6TipFpFhVnz6KOBrUjKw8osLDOK+PLXJkjAke/q7BfTSWAKki0gVPkhgLXOm7g4ikqup67+b5wHoAVT3VZ58HgYONIVFUVCrvLd/OsB5xtGpmM8waY4KHY8lCVctFZAIwFwgHXlLV1SLyMJCpqrOACSIyHCgDvgfGORVPQ1iwoZCCAyXWBGWMCTpO1ixQ1TnAnGplD/g8vt2PczxY/5E5Y0ZWHjFNIsjoGe92KMYYU69sKtR6UlxWwYerdjKiTweaRoa7HY4xxtQrSxb15NO1BRwsKbcmKGNMULJkUU9mZOURH9OEoce1czsUY4ypd5Ys6sHeolLmZRcwsn8C4TbDrDEmCFmyqAdzVu6krEKtCcoYE7QsWdSDGVl5dI1rTu+Elm6HYowxjrBkcYzy9v7A4k17GD0gERFrgjLGBCdLFsdoVtZ2AEbZIkfGmCBmyeIYzczKY1BKa1LaNXM7FGOMcYwli2Pw3c79fLfzgHVsG2OCniWLYzBj2XbCw4Tz+9oMs8aY4GbJ4ihVViqzsvI4LTWWdi2auB2OMcY4ypLFUVqyeQ/b9xVbE5QxJiRYsjhKM7K20ywqnLN6tXc7FGOMcZwli6NQWl7JnJU7OLtXe5pFOTrLuzHGBARLFkdhXnYB+34oY5Q1QRljQoQli6MwM2s77ZpHcWq3WLdDMcaYBmHJ4ggdKC7jk7X5XNCvIxHh9vIZY0KDfdodoQ9X7aSkvNKaoIwxIcWSxRGambWdTu2aMTC5tduhGGNMg3E0WYjICBHJFpEcEbm3hud/LSIrRSRLRL4SkV7e8rNEZKn3uaUikuFknP4q2F/MNxt2M6p/gs0wa4wJKY4lCxEJByYB5wK9gCuqkoGPN1S1r6oOAB4DJnrLdwMXqmpfYBzwqlNxHolZy7dTqVgTlDEm5DhZsxgC5KjqRlUtBaYCo3x3UNX9PpvNAfWWL1PV7d7y1UC0iLg+p8bMrO30TWxF17gWbodijDENyslkkQhs89nO9Zb9hIjcIiIb8NQsbqvhPJcA36pqiSNR+mnDroOszNvHqAEJboZhjDGucL2DW1UnqWpX4B7gD77PiUhv4FHgxpqOFZHxIpIpIpm7du1yNM6Zy/IIExjZ35KFMSb0OJks8oBkn+0kb9nhTAVGV22ISBLwLnCtqm6o6QBVnayqaaqaFhcXVw8h10xVmZG1nZO6xhLfsqlj1zHGmEDlZLJYAqSKSBcRiQLGArN8dxCRVJ/N84H13vLWwGzgXlX92sEY/bJs21627imyJihjTMhyLFmoajkwAZgLrAWmq+pqEXlYREZ6d5sgIqtFJAu4E8+dT3iP6wY84L2tNktE4p2KtS4zl+XRJCKMEX06uBWCMca4ytEpU1V1DjCnWtkDPo9vP8xxjwCPOBmbv8oqKnl/xQ6GH9+emKaRbodjjDGucL2DO9B9lbObwkOl1gRljAlplizqMHNZHq2iIxnWw7VWMGOMcZ0li1oUlZbz0Zp8zuvbkagIe6mMMaHLPgFr8fGafIpKKxhtTVDGmBBnyaIWM5blkdCqKemd27odijHGuMqSxWEUHizhi/W7GTkgkbAwm2HWGBPaLFkcxuyVO6ioVEYPtCYoY4yxZHEY7y7Lo2eHGHp2aOl2KMYY4zpLFjXYUniIZVv3MmqArVthjDFgyaJGM7M8S2mMtLugjDEGsGTxM54ZZvMY0qUtia2j3Q7HGGMCgiWLalbl7WfjrkOMtiYoY4z5kSWLamZk5REZLpzX12aYNcaYKpYsfFRUKu8t386wHvG0bhbldjjGGBMwLFn4WLChkIIDJdYEZYwx1Viy8DEjK48WTSI483ibYdYYY3xZsvAqLqvgw1U7GdGnA00jw90OxxhjAoolC69P1xZwsKTcmqCMMaYGliy8ZmTlER/ThBO7tnM7FGOMCTiWLIC9RaXMyy7gwv4JhNsMs8YY8zOWLIA5K3dSVqHWBGWMMYfhaLIQkREiki0iOSJybw3P/1pEVopIloh8JSK9fJ77vfe4bBE5x8k4Z2TlcVxcc/ok2gyzxhhTE8eShYiEA5OAc4FewBW+ycDrDVXtq6oDgMeAid5jewFjgd7ACOAZ7/nqXd7eH1i8aQ+jByQiYk1QxhhTEydrFkOAHFXdqKqlwFRglO8OqrrfZ7M5oN7Ho4CpqlqiqpuAHO/56t0PpeUMPz6eUTbDrDHGHFaEg+dOBLb5bOcCJ1TfSURuAe4EooAMn2MXVjv2Zx0KIjIeGA+QkpJyVEF2i4/hxXHpR3WsMcaECtc7uFV1kqp2Be4B/nCEx05W1TRVTYuLi3MmQGOMMY4mizwg2Wc7yVt2OFOB0Ud5rDHGGAc5mSyWAKki0kVEovB0WM/y3UFEUn02zwfWex/PAsaKSBMR6QKkAosdjNUYY0wtHOuzUNVyEZkAzAXCgZdUdbWIPAxkquosYIKIDAfKgO+Bcd5jV4vIdGANUA7coqoVTsVqjDGmdqKqde/VCKSlpWlmZqbbYRhjTKMiIktVNa2u/Vzv4DbGGBP4LFkYY4ypkyULY4wxdQqaPgsR2QVsOYZTxAK76ymcxs5ei5+y1+N/7LX4qWB4PTqpap0D1YImWRwrEcn0p5MnFNhr8VP2evyPvRY/FUqvhzVDGWOMqZMlC2OMMXWyZPE/k90OIIDYa/FT9nr8j70WPxUyr4f1WRhjjKmT1SyMMcbUKaSThYgki8jnIrJGRFaLyO1ux+Q2EQkXkWUi8r7bsbhNRFqLyNsi8p2IrBWRE92OyU0i8hvv38kqEXlTRJq6HVNDEpGXRKRARFb5lLUVkY9FZL333zZuxuikkE4WeCYp/K2q9gKGArfUsPRrqLkdWOt2EAHin8CHqtoT6E8Ivy4ikgjcBqSpah88k4OOdTeqBvcfPMs8+7oX+FRVU4FPvdtBKaSTharuUNVvvY8P4Pkw+NmKfKFCRJLwTBX/otuxuE1EWgGnAf8GUNVSVd3rblSuiwCiRSQCaAZsdzmeBqWqXwB7qhWPAl7xPn6F/63JE3RCOln4EpHOwEBgkbuRuOpJ4HdApduBBIAuwC7gZW+z3Isi0tztoNyiqnnA48BWYAewT1U/cjeqgNBeVXd4H+8E2rsZjJMsWQAi0gL4L3CHqu53Ox43iMgFQIGqLnU7lgARAQwCnlXVgcAhgriJoS7etvhReJJoAtBcRK52N6rAop5bS4P29tKQTxYiEoknUbyuqu+4HY+LTgZGishmPEvcZojIa+6G5KpcIFdVq2qab+NJHqFqOLBJVXepahnwDnCSyzEFgnwR6Qjg/bfA5XgcE9LJQkQET5v0WlWd6HY8blLV36tqkqp2xtNx+Zmqhuw3R1XdCWwTkR7eojPxrNwYqrYCQ0Wkmffv5kxCuMPfxyy8K3x6/53pYiyOCulkgefb9DV4vkVneX/OczsoEzBuBV4XkRXAAOAvLsfjGm8N623gW2Alns+OkBm9DCAibwILgB4ikisi1wN/A84SkfV4al9/czNGJ9kIbmOMMXUK9ZqFMcYYP1iyMMYYUydLFsYYY+pkycIYY0ydLFkYY4ypkyULE3JEREXkHz7bd4nIg/V8jet8bscuFZGV3sdHfGuld3bkafUZnzFHym6dNSFHRIrxzG+Urqq7ReQuoIWqPujQ9Tbjma11txPnN6YhWM3ChKJyPAPKflP9CRH5j4hc6rN90PvvMBGZLyIzRWSjiPxNRK4SkcXeWkNXfy8uIrEiMktEVojINyLSx1v+iIi8IiILvesj/NJb3k1EsryPI0TkCe+aEitE5GZv+d+967KsEJFHj+XFMaYmEW4HYIxLJgErROSxIzimP3A8nmmqNwIvquoQ76JZtwJ3+HmePwGLVHWkiJyNZ52ENO9zffHMudQS+FZEZlc79iY8E/n1V9UK7+I77YHzgN6qqiLS+gh+J2P8YjULE5K8swtPwbOgj7+WeNdAKQE2AFVTdK8EOh/BeU4BXvXG8RGQ4DP9+QxVLVbVAuALIL3ascOB51S1wnv8HjzJqxJ4QUQuwjNDrjH1ypKFCWVPAtcDvutUlOP9uxCRMCDK57kSn8eVPtuV1F8tvXonYp2dit5ZYNOAGXgW36leGzHmmFmyMCHL+618Op6EUWUzMNj7eCQQ6cClvwSuAhCR4UCeqlbVBkaLSBMRiQNOBTKrHfsx8GsRCfce31ZEYoCWqvo+nn6YgQ7EbEKc9VmYUPcPYILP9gvATBFZDnyIM006DwAveWezPQhc5/PcKmA+0A74P1XN9yaDKs8DqXj6W8qBZ4H3gXdEpAmeL4B3OhCzCXF266wxAUJEHgF2q+qTbsdiTHXWDGWMMaZOVrMwxhhTJ6tZGGOMqZMlC2OMMXWyZGGMMaZOliyMMcbUyZKFMcaYOlmyMMYYU6f/B6SzoO3VIu3IAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Compute a list of LDA Mallet Models and corresponding Coherence Values\n", "def compute_coherence_values(dictionary, corpus, texts, limit, start=2, step=3):\n", " coherence_values = []\n", " model_list = []\n", " for num_topics in range(start, limit, step):\n", " model = gensim.models.wrappers.LdaMallet(mallet_path, corpus=corpus, num_topics=num_topics, id2word=id2word)\n", " model_list.append(model)\n", " coherencemodel = CoherenceModel(model=model, texts=texts, dictionary=dictionary, coherence='c_v')\n", " coherence_values.append(coherencemodel.get_coherence()) \n", " return model_list, coherence_values\n", "model_list, coherence_values = compute_coherence_values(dictionary=id2word, corpus=corpus, texts=data_lemmatized,\n", " start=2, limit=12, step=1)\n", "\n", "# Visualize the optimal LDA Mallet Model\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "limit=12; start=2; step=1;\n", "x = range(start, limit, step)\n", "plt.plot(x, coherence_values)\n", "plt.xlabel('Num Topics')\n", "plt.ylabel('Coherence score')\n", "plt.legend(('coherence_values'), loc='best')\n", "plt.show()" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "hidden": true, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Num Topics = 2 has Coherence Value of 0.3009\n", "Num Topics = 3 has Coherence Value of 0.3654\n", "Num Topics = 4 has Coherence Value of 0.3877\n", "Num Topics = 5 has Coherence Value of 0.3953\n", "Num Topics = 6 has Coherence Value of 0.4238\n", "Num Topics = 7 has Coherence Value of 0.3779\n", "Num Topics = 8 has Coherence Value of 0.4006\n", "Num Topics = 9 has Coherence Value of 0.391\n", "Num Topics = 10 has Coherence Value of 0.4349\n", "Num Topics = 11 has Coherence Value of 0.3763\n" ] } ], "source": [ "# Print the coherence scores\n", "for m, cv in zip(x, coherence_values):\n", " print('Num Topics =', m, ' has Coherence Value of', round(cv, 4))" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "With our models trained, and the performances visualized, we can see that the optimal number of topics here is **10 topics** with a Coherence Score of **0.43** which is slightly higher than our previous results at 0.41. However, we can also see that the model with a coherence score of 0.43 is also the highest scoring model, which implies that there are a total 10 dominant topics in this document.\n", "\n", "We will proceed and select our final model using 10 topics." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hidden": true, "scrolled": true }, "outputs": [], "source": [ "# Select the model with highest coherence value and print the topics\n", "optimal_model = model_list[8]\n", "model_topics = optimal_model.show_topics(formatted=False)\n", "pprint(optimal_model.print_topics(num_words=10)) # Set num_words parament to show 10 words per each topic" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "By using our **Optimal LDA Mallet Model** using Gensim's Wrapper package, we displayed the 10 topics in our document along with the top 10 keywords and their corresponding weights that makes up each topic.\n", "\n", "Note that output were omitted for privacy protection. However the actual output is a list of the 10 topics, and each topic shows the top 10 keywords and their corresponding weights that makes up the topic." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true, "hidden": true }, "source": [ "## 9.1 Visual the Optimal LDA Mallet Model" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hidden": true }, "outputs": [], "source": [ "# Wordcloud of Top N words in each topic\n", "from matplotlib import pyplot as plt\n", "from wordcloud import WordCloud, STOPWORDS\n", "import matplotlib.colors as mcolors\n", "cols = [color for name, color in mcolors.TABLEAU_COLORS.items()]\n", "cloud = WordCloud(stopwords=stop_words,\n", " background_color='white',\n", " width=2500,\n", " height=1800,\n", " max_words=10,\n", " colormap='tab10',\n", " color_func=lambda *args, **kwargs: cols[i],\n", " prefer_horizontal=1.0)\n", "topics = optimal_model.show_topics(formatted=False)\n", "fig, axes = plt.subplots(2, 5, figsize=(10,10), sharex=True, sharey=True)\n", "for i, ax in enumerate(axes.flatten()):\n", " fig.add_subplot(ax)\n", " topic_words = dict(topics[i][1])\n", " cloud.generate_from_frequencies(topic_words, max_font_size=300)\n", " plt.gca().imshow(cloud)\n", " plt.gca().set_title('Topic ' + str(i), fontdict=dict(size=16))\n", " plt.gca().axis('off')\n", "plt.subplots_adjust(wspace=0, hspace=0)\n", "plt.axis('off')\n", "plt.margins(x=0, y=0)\n", "plt.tight_layout()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Here we also visualized the 10 topics in our document along with the top 10 keywords. Each keyword's corresponding weights are shown by the size of the text.\n", "\n", "Note that output were omitted for privacy protection." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 10 Analysis" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Now that our **Optimal Model** is constructed, we will apply the model and determine the following:\n", "- Determine the dominant topics for each document\n", "- Determine the most relevant document for each of the 10 dominant topics\n", "- Determine the distribution of documents contributed to each of the 10 dominant topics" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true, "hidden": true }, "source": [ "## 10.1 Finding topics for each document" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hidden": true, "scrolled": true }, "outputs": [], "source": [ "def format_topics_sentences(ldamodel=optimal_model, corpus=corpus, texts=data):\n", " sent_topics_df = pd.DataFrame()\n", " # Get dominant topic in each document\n", " for i, row in enumerate(ldamodel[corpus]): \n", " row = sorted(row, key=lambda x: (x[1]), reverse=True) \n", " # Get the Dominant topic, Perc Contribution and Keywords for each document\n", " for j, (topic_num, prop_topic) in enumerate(row):\n", " if j == 0: \n", " wp = ldamodel.show_topic(topic_num) \n", " topic_keywords = \", \".join([word for word, prop in wp])\n", " sent_topics_df = sent_topics_df.append(pd.Series([int(topic_num), round(prop_topic,4),\n", " topic_keywords]), ignore_index=True)\n", " else:\n", " break\n", " sent_topics_df.columns = ['Dominant_Topic', 'Perc_Contribution', 'Topic_Keywords'] # Create dataframe title\n", " # Add original text to the end of the output (recall that texts = data_lemmatized)\n", " contents = pd.Series(texts)\n", " sent_topics_df = pd.concat([sent_topics_df, contents], axis=1)\n", " return(sent_topics_df) \n", "df_topic_sents_keywords = format_topics_sentences(ldamodel=optimal_model, corpus=corpus, texts=data)\n", "df_dominant_topic = df_topic_sents_keywords.reset_index()\n", "df_dominant_topic.columns = ['Document_No', 'Dominant_Topic', 'Topic_Perc_Contrib', 'Keywords', 'Document']\n", "df_dominant_topic.head(10)" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Note that output were omitted for privacy protection. However the actual output is a list of the first 10 document with corresponding dominant topics attached." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true, "hidden": true }, "source": [ "## 10.2 Finding documents for each topic" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hidden": true, "scrolled": true }, "outputs": [], "source": [ "# Group top 10 documents for the 10 dominant topic\n", "sent_topics_sorteddf_mallet = pd.DataFrame()\n", "sent_topics_outdf_grpd = df_topic_sents_keywords.groupby('Dominant_Topic') \n", "for i, grp in sent_topics_outdf_grpd:\n", " sent_topics_sorteddf_mallet = pd.concat([sent_topics_sorteddf_mallet,\n", " grp.sort_values(['Perc_Contribution'], ascending=[0]).head(1)], axis=0)\n", "sent_topics_sorteddf_mallet.reset_index(drop=True, inplace=True)\n", "sent_topics_sorteddf_mallet.columns = ['Topic_Num', \"Topic_Perc_Contrib\", \"Keywords\", \"Document\"]\n", "sent_topics_sorteddf_mallet " ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Note that output were omitted for privacy protection. However the actual output is a list of most relevant documents for each of the 10 dominant topics." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true, "hidden": true }, "source": [ "## 10.3 Document distribution across Topics" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Dominant TopicNum_DocumentPerc_Document
00.0270.0528
11.0270.0528
22.0230.0450
33.0960.1879
44.0550.1076
55.0430.0841
66.0260.0509
77.0420.0822
88.0440.0861
99.01280.2505
\n", "
" ], "text/plain": [ " Dominant Topic Num_Document Perc_Document\n", "0 0.0 27 0.0528\n", "1 1.0 27 0.0528\n", "2 2.0 23 0.0450\n", "3 3.0 96 0.1879\n", "4 4.0 55 0.1076\n", "5 5.0 43 0.0841\n", "6 6.0 26 0.0509\n", "7 7.0 42 0.0822\n", "8 8.0 44 0.0861\n", "9 9.0 128 0.2505" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Number of Documents for Each Topic\n", "topic_counts = df_topic_sents_keywords['Dominant_Topic'].value_counts()\n", "topic_contribution = round(topic_counts/topic_counts.sum(), 4)\n", "topic_num_keywords = {'Topic_Num': pd.Series([0.0,1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0])}\n", "topic_num_keywords = pd.DataFrame(topic_num_keywords)\n", "df_dominant_topics = pd.concat([topic_num_keywords, topic_counts, topic_contribution], axis=1)\n", "df_dominant_topics.reset_index(drop=True, inplace=True)\n", "df_dominant_topics.columns = ['Dominant Topic', 'Num_Document', 'Perc_Document']\n", "df_dominant_topics" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Here we see the number of documents and the percentage of overall documents that contributes to each of the 10 dominant topics." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "# 11 Answering the Questions" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Based on our modeling above, we were able to use a very accurate model from Gibb's Sampling, and further optimize the model by finding the optimal number of dominant topics without redundancy.\n", "\n", "As a result, we are now able to see the 10 dominant topics that were extracted from our dataset. Furthermore, we are also able to see the dominant topic for each of the 511 documents, and determine the most relevant document for each dominant topics.\n", "\n", "With the in-depth analysis of each individual topics and documents above, the Bank can now use this approach as a \"Quality Control System\" to learn the topics from their rationales in decision making, and then determine if the rationales that were made are in accordance to the Bank's standards for quality control." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": false, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": { "height": "413px", "left": "1097px", "top": "110px", "width": "183px" }, "toc_section_display": false, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }