{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "In this notebook, I want to wrap up some loose ends from last time." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The two cultures" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This \"debate\" captures the tension between two approaches:\n", "\n", "- modeling the underlying mechanism of a phenomena\n", "- using machine learning to predict outputs (without necessarily understanding the mechanisms that create them)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"One" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "I was part of a research project (in 2007) that involved manually coding each of the above reactions. We were determining if the final system could generate the same ouputs (in this case, levels in the blood of various substrates) as were observed in clinical studies. \n", "\n", "The equation for each reaction could be quite complex:\n", "\"reaction\n", "\n", "This is an example of modeling the underlying mechanism, and is very different from a machine learning approach.\n", "\n", "Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2391141/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The most popular word in each state" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"The\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A time to remove stop words" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Factorization is analgous to matrix decomposition" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "### With Integers" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Multiplication: \n", "\t$$2 * 2 * 3 * 3 * 2 * 2 \\rightarrow 144$$\n", " \n", "\"factorization\"\n", "\n", "Factorization is the “opposite” of multiplication: \n", "\t $$144 \\rightarrow 2 * 2 * 3 * 3 * 2 * 2$$\n", " \n", "Here, the factors have the nice property of being prime.\n", "\n", "Prime factorization is much harder than multiplication (which is good, because it’s the heart of encryption)." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "### With Matrices" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Matrix decompositions are a way of taking matrices apart (the \"opposite\" of matrix multiplication).\n", "\n", "Similarly, we use matrix decompositions to come up with matrices with nice properties." ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Taking matrices apart is harder than putting them together.\n", "\n", "[One application](https://github.com/fastai/numerical-linear-algebra/blob/master/nbs/3.%20Background%20Removal%20with%20Robust%20PCA.ipynb):\n", "\n", "\"The\"" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "What are the nice properties that matrices in an SVD decomposition have?\n", "\n", "$$A = USV$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Some Linear Algebra Review" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matrix-vector multiplication" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$Ax = b$ takes a linear combination of the columns of $A$, using coefficients $x$\n", "\n", "http://matrixmultiplication.xyz/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matrix-matrix multiplication" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$A B = C$ each column of C is a linear combination of columns of A, where the coefficients come from the corresponding column of C" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"NMF\n", "\n", "(source: [NMF Tutorial](http://perso.telecom-paristech.fr/~essid/teach/NMF_tutorial_ICME-2014.pdf))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Matrices as Transformations" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The 3Blue 1Brown [Essence of Linear Algebra](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab) videos are fantastic. They give a much more visual & geometric perspective on linear algreba than how it is typically taught. These videos are a great resource if you are a linear algebra beginner, or feel uncomfortable or rusty with the material.\n", "\n", "Even if you are a linear algrebra pro, I still recommend these videos for a new perspective, and they are very well made." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAUDBA0ICQoLCBAICwoOCgoKCwoLDwoIDgoKCAgICAoKCgsKDxALCggOCgoKDhUNDhERExMTCgsWGBYSGBASExIBBQUFCAcIDQkJDxYTEBISFRIaHhMbHhMTHhISFRUVEhIZHh4SEhYVFRISFRUVEhISEhUSFRUVFRIVEhUVFRUVFf/AABEIAWgB4AMBIgACEQEDEQH/xAAdAAEAAwEBAQEBAQAAAAAAAAAABAUGCAcDAgEJ/8QAWhAAAgECAgQJCAQLBQQJAQkAAQIDAAQFEQYSITEHExQWQVFTcZIVIlJhkdHS0wiBoaMyNENUYmNypLHh8QkjQpPBJDNzszU2N3R1grS18BcYhZSissLE1OL/xAAZAQEAAwEBAAAAAAAAAAAAAAAAAQIDBAX/xAA7EQABAgIHBQYGAgICAgMAAAABAAIDERITITFRYZEEUoGh0TJBcbHB4RQiM9Li8GKSBSOy8XKiJEJj/9oADAMBAAIRAxEAPwDjKlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlKURKUpREpSlESlT+QJ21t7Lj5VOQJ21t7Lj5VWoFZ1rc9D0UClT+QJ21t7Lj5VOQJ21t7Lj5VKBStbnoeigUqfyBO2tvZcfKpyBO2tvZcfKpQKVrc9D0UClT+QJ21t7Lj5VOQJ21t7Lj5VKBStbnoeigUqeMPTtrX2XHyqsJdFXWPjNe3aP0k46Ud/8AdxnZ66kQ3G5VdtDG3mXA9FQUqfyBO2tvZcfKpyBO2tvZcfKqKBVq1ueh6KBSp/IE7a29lx8qnIE7a29lx8qlApWtz0PRQKVP5AnbW3suPlU5AnbW3suPlUoFK1ueh6KBSp/IE7a29lx8qnIE7a29lx8qlApWtz0PRQKVP5AnbW3suPlU5AnbW3suPlUoFK1ueh6KBSp/IE7a29lx8qvvZYMJm1Y5rQt0AmZM/UNeMAn1CpDCVBjNAmZ6HoqmlXOI6Pm3OU0kCHozFwQe5hEVP1GovIE7a29lx8qhhkWFGxmETHkeigUqfyBO2tvZcfKpyBO2tvZcfKqKBU1rc9D0UClT+QJ21t7Lj5VOQJ21t7Lj5VKBStbnoeigUqfyBO2tvZcfKpyBO2tvZcfKpQKVrc9D0UClT+QJ21t7Lj5VOQJ21t7Lj5VKBStbnoeigUqfyBO2tvZcfKpyBO2tvZcfKpQKVrc9D0UClXthoy1wCYZLVwN4BlzGXWhj1vs21ElwwISry26sN6stypHeDFmKkw3C1VEdhJANoyPRVtKn8gTtrb2XHyqcgTtrb2XHyqigVatbnoeigUqfyBO2tvZcfKpyBO2tvZcfKpQKVrc9D0UClT+QJ21t7Lj5VOQJ21t7Lj5VKBStbnoeigUqfyBO2tvZcfKpyBO2tvZcfKpQKVrc9D0UClT+QJ21t7Lj5VOQJ21t7Lj5VKBStbnoeigUqwGHofy1r7Lgb/WYtgqfcaLOkfGF7cx79dOOlGXWTHGcl9Z2VIhuNyqY7BKZvyPRUFKn8gTtrb2XHyqcgTtrb2XHyqigVatbnoeigUqfyBO2tvZcfKpyBO2tvZcfKpQKVrc9D0UClT+QJ21t7Lj5VOQJ21t7Lj5VKBStbnoeigUqfyBO2tvZcfKpyBO2tvZcfKpQKVrc9D0UClT+QJ21t7Lj5VOQJ21t7Lj5VKBStbnoeigUqfyBO2tvZcfKr72WCcc2rHNaFugEzIT3a0YzPqFSGEqDGaBM+R6KppXoNzoFFeo0uBTJcKBrNaSkRXEY6sjkHHryUdRasLf2b27mOdZI5BvRwUYd4bbVFqvhSlKIlKUoiUpSiJUvDMSktm1oWZesbwf2lOw1EpUgkWhQ5ocJEWLS+ULe82XS8RKfy8X4JPW6f17xUPE9HZIl149WaHeJI/OGXrA2j7R66pqmYZiUls2tCzL1jeD+0p2GtKYd2hxF/uuepcz6Z4G0cO8cxkodK0vlC2vNl0vESn8tEPNJ63T/AF294qHiejkkS68erNFvEkXnjL9IDaPtHrqDDN7bQpbtAnReKJzu4G71yVNSlKzXQlKUoiUpSiJXpOhPAXjONWkN3hlsk9vKJDG/KbCFjxM0sD5xTzpKuTxvvXaBmNhrzarHRmdo7y1eMsrrcQurKSpVkmRlZSNoYEA50Re24d9H/SiFdSWxWaHLbHLdYWwAHUxuc1y+sDqrLcIvBHcYUIGvUSxknMgihee0u1kaERlwr2ssmp/vEy1ss89g2V1V/aNTsmjdoqllV8WgVwCQHVbLEJArgfhKHVWyPSqnoFcAVoIhudaFzu2cTpMNE5XcRcfPNTMTwyS2bKZWXqO8HuYbDUOrnDNIpIl1JNWaLcY5fPGXqJ2j7R6qmcgtrzbatxEvYynzSepH7+/uFTQDuyeBv91Fc5n1BxFo6jmM1BsNGLu4tZbuC3vpbSIkTXUcM0kMJUBiJZlUxxkBlJ1iN466qK614L/pGQaN6MDCb6yumu4o7qKBkELWtyLmSaYNO7OHTbKwZVSTWC55jWyXkqsyCLCuhrg4TFoSlKVClKUpREpSlEX7hlKMGQsrDaGBII7iKvodIRMAl+izLuEi5JIvTsIyDDds2evOs9SrteW3LOJBa+8W6FaGbR0TKXsHWZRtMZySRe8HLPp27PVnVDNGUJVwysNhUgqR3g7RSGUoQyFlYbmUlSO4jaKvotIhMAl+izLuEi5JIvcRkG6Nmz151b5HZHl7c1l/th/yGjuh5LPUrQzaOiYF7B1mXeY2ySRe8HIHp27PVnVDNEUJVwysN6sCpHeDtFUcwtvWsOM193Qr8UpSqrRKUpREpSlESpuF4pJbNnCzL1rvVu9TsPfvqFSpBItCq5ocJETC0nLba9/GF5PKfysYzRj+mnRmT/OoeKaPSQrrrlLDlmJY/PGXWQNq9+711T1NwvFJLZs4WZetd6t+0p2Hv31pTDu0OIWFU5n0zZgbuBvHMZKFStJy22vPxheTzH8rGM0Y9bp0bT/OoeKaPSQjXXVli3iWLzxl1nLavfu9dQYZvFoUt2gTovFE5+huPnkqelKVmuhKUpREpSlESlKURfS2naJ1eJnRwc1dCUZT1qw2g91bmw4QhcIsOOQx3sQ2LMMop4h1rIuWt0bM1J6SapOZ0/6nxj3U5nT/AKnxj3VrUvwK5/i4O8Ff3OgMd6rS4DMtyoGbWsuUM8Y7jkrjft80dRasLfWbwO0cyvHIpyZHBQjvB21f22i1zEweNo0cHNXWTUZT1hhtB7q3NliU1xGIcbitb2IbFmDLFcR+tZBkG7s1J6SaVL8CnxcHeC8gpW+0l0Ej1s8MlaRT+RmGq6DL01Go+Xcv11SNoZcA5HiQeotl/pUVL8CnxcHeCzlK0XM6f9T4x7qczp/1PjHuqal+BT4uDvBZ2laLmdP+p8YpzOn/AFPjFKl+BT4uDvBZ2laLmdP+p8YpzOn/AFPjFKl+BT4uDvBZ2pmGYlJbNrQsy9Y3g96nYatuZ0/6nxinM6f9T4xUiFEFoBVXbTAcJFwIX78oW15sul4iU/loh5pPW6f67e8VDxPRySJdePVmh3iSPzhl6wNo+0eupPM6f9T4xUvDMBu7ZtaFo16xrgg/tKRka0oOd2mnxH7aueths+m8eBMx1HMZLJUr0ObARdA8qSOKXtYWBDHrZD/M+sVRT6FTBjqGJ16Gz1c+8HcfbVHbM8XCa1h7fCdY4yPLX9OSzNK0XM6f9T4xTmdP+p8YqtS/ArT4uDvBZ2puBfjMH/Gi/wCYtWvM6f8AU+MVKwfRKdZ4SeJyE0Z/CHRItKl+BU/Fwd4Ls/8AtH/+rtj/AOLw/wDt+JVwJX+g39oThz3OAWKRaut5WibziFGQw/Eek99cN8zp/wBT4xVWw3OEwFZ+0Q2GTnAFZ2pmCYe13cQW8ZjV5ZooUaRhGitNIsamRzsSMFgSx3DM1bczp/1PjFfwaHzndxHjFWqX4FV+Lg7wXQOjv0YsaNuXNxo7PahSWRp7i5GqgzYI0duy6wHRrZV4JDhtvfsotWFvMxCiGU+YWYgebJ0DM+vuFde/QEw+S2wrHY5SMuMidVVtYAtazqx9RIVfCOquPE0QnIB/udoz/DHTV20z8pE5LF9SJPa6jPC48Lj55r2vB/oeYvdLrR3Gj+rnkSLiebI9Izht2XWy6M68p0Z4Nbi8hkuZpLCwsVkkhF9fym2immiOTQ2yIr3N1KBmSIYnC5ecV2Z9P/2d9nPbT4vHOwKcRZFVDa4BWS7HeMgTl3mvE+G/DvKOITLbNxNvas9hZWzENHHaWUjwxldXzlkmKtM7NrFpJpCaoILiSALlqdrhtaHOIkTKy0KF/wDQS8ubCa/wS4wnGLeHbMlhJObiIBC5L2tzDFJnkGIUZswByBryOutP7PnDriyx69jkIEMmFyOyKwIaWC9sRExG/NUmmA/bavF+GHQmSPHsYWAQJEMUvxEgYKFi5ZMY1Ay2AIVGXqqKp05SV/iIdGlSEliNE9HLnFrqK0w6KW4uZDkkUYzJ62YnJUjUbWdiFUAkkAZ16Ni3A/bYZKbbG8WwazvRkGtYo7/FOJYgf3dzLawmKCYNsKZtlsNdCcCmivM/QbEMYQIuLXdseLnOq3EJPOLSwWMtmoj15EuG2ecWQNnxa5chTaLzuzM7QszEszNIGLMxzLMTtLE7czUCE43BH7RDZKk4CdvBXnCjwTXuj8VvcXHJrmwuADbYlZObq2mJUuEEmqrJIVBIV1XWCvq62o2WBrtH6HeDNjGj2PaP4nqPbkRyW7awm4lr5JwWjUgiPibi2inXo15GOWeefgPAngS2GPYZcYuIxaQ3kUs4z1yhiYsjOq7WRJgjMozzCtsO4qp05SQ7RDABpCRXxTgnFmkTaRXuHYO0iB1tJlur68VZFV42ns7ON2tUdDmOOZG3DVqTj3AddLhz4pg01jjOGpnxs1gZeOtwqqzG6s5kSeEqCGKgMVTJ21VzI2/0wMFw6+xaO40bMDtJG8t/KkjNFJcySlldWcnOcqSX1fN/A/xa1W/0CBcYfpFJAWTiLmymWSNXDAyWxSaGQqN7qONUHoEr9dTVPlOSDaYRdRDhNcvQylGDIWVhuZSVI7iNorobgw4D8Q0msoJy+BSxyIXTjLh4rmJBIyEyJBDIy7V3HLPZnnnWT+kHwd8j0jxWGxWGO2F0XijzVBGtxHHc8WigZLGplKqvQFAq1+ijo9Na6W4PIxjAE8qnVYE5PZXSEZdIIJH11ZrYjRMCzksoj4D3USROcsDPzVdj3AFdwYpeWEcthlZiBr67mkaC3s0u7eGeFpZnQM2vxoRVWPXdgwVGCsRC4S+Aq/wPDLfFHa0u8Om4vK4tuUji+OGcTSxXUMMqwyH8CQKVOabtdQfY/p1JK+LLaYflHG3FYpdtxhQy3z2qYZCSOhYbK0QLl03M/pHPFYzwnYriWCw4LiYsZLGNIIzJETHcSpaavELM5YxuEKRnNVRiUUsW25yIRfKQl5Kp2lkKYc6ctfDuHqufaVrsS0JYH/ZmVl9FyFZfrHmt37KhnQ2f9T4h7qgwHjuV27bBInSHFZ2laLmdP+p8YpzOn/U+MVFS/Aqfi4O8FnaVouZ0/wCp8YpzOn/U+MUqX4FPi4O8FnaVouZ0/wCp8YpzOn/U+MUqX4FPi4O8Fnam4Xikts2cLMvWu9W71Ow9++rXmdP+p8YpzOn/AFPjFSIUQWgFQ7aYDhIuBC/XLLa8/GF5PMfyse1GJ6XTo2n+dQ8U0ekgXXXVlh3iWPzxl1kDaP4eupXM6f8AU+MVNwvAry2bOFo1611wVbvUjI9++tKDndpp8Queuhs+m8SwJs4G8cxkshSvQp9HxdAm4SOGbtIWBVj1tGfX1bfXVHNoXOCdQwsvQ2tq594I2GqO2Z4uE1rD2+E68yP73rM0rRczp/1PjFOZ0/6nxiq1L8CtPi4O8FnaVouZ0/6nxinM6f8AU+MUqX4FPi4O8FnaUpWS6UpSlEX1tbhonV4mdHBzV0JRlPWrLkQe6txYcIIuEEOOQx3sYGQmAEdxGD0rIMtbo2ZqT0k1gqURegXOgMd6hlwGZblQM2tZSIZ4/qOSuN+3JRs2a1Ya+s3gcxzrJG43o4KMO8Ntr8W07RMHjLo4OaupKMp6ww2g91bqx4QhcoIcciS9iGxZhlFPH61dcg3RszXPpJoiwNK9AutAo71WlwGZLlctY2kpENxGOrJslcDbt80dWtWGvrN4HaOdXjkGxkcFGHeG20RfClKURKUpREpSlESpmG4lJbNnCzL1jeD+0p2GodKkEi0KHNDhIiYWk5fbXmy6XiJe2iHmknpdP9dveKiYno7JEuvHqzQ7xJF5wy62A2j7R66pqmYZicls2tCzL1jeD3qdhrSmHdocR+2rnqXM+meBtHUcxkodTcC/GYP+NF/zFq48oW15sul4iU/l4vwSet0/r3itzwfcAmL4m9nc4dAtxYyTKUvRLbxx6kNyYpXYPIJPMaNwQFJOqctbZVXMkJgzCvDi0jRcCDy1u9cl07/aP/8AV2x/8Xh/9vxKuBK/0g+m7oJe6QYJawYRC1zNHiUU7xq0URES2d7EXHGsobJ5IxkCT52eWQJHHS/Rs0iJy5BP9ctmPtM2Qqi2XkyKWICgkkgADaSTsAAG816j9KS05HjvIclAssNwiwzGXnGDCLNnY5f4uMkcE+qvevo1/RQuLO/gxHSXiEEDpPBYRuJ2a4jbXRrqSPOIRRsFYJGz65GTEKCr+C/SF4zF9M8VitVaSeXFTYwxjJS80UiYckYLZAEyRhQSQPXlRF0B/Z2/9FaQftwf+lua4sr/AEF+hTwZYhgWG4tHi8PJpLiWPiY2eKQlY7eVC7cSzhV1pANu3zTs3Z8W6fcFuJ4DJbRYtbvbvcO6W4LwTcc0TQq4VoXddhmj2kgeeKIui/7NL8axv/gWX/Nuq5e4Qf8ApXEf+/Xf/qpa7V+gpwWYlo9NismMwNarNHaRwhngkMhie5aTIQu5VVDJtbLPWGWeRy5h4euCfE8Fur28xG3aKzkxGdIbkPBIspnluJ4tURuXGtGjN5ygjLI5HZRF6b/Z2YhI+kV1G7MyDCLgqG87I8vwwbCdoGXRnlXnPD3hkN1pJjYik4q48p3gMcuxXblEm1G6Cerb3V7t9BrgjxTBcYuL7FLdra2fDJII3Z4HMkk91YzoFSN2cDi4nJJAA2DpryX6UvBJilniuM4rPbsMNe+lnW7V4HXi7y5CxZoH41WLyKpBXYfVtq4fjasnQrPlNHvs9cV0npnj01jwYWtzhxjWeDDcHjJaOC8VWgnsbO4BjmSSJtVhJmSpyyJ2EZ1yB/8AXXF+2sv/AMBg/wD/AFa9Q+ipww2qWF1o5pM4TDbpZY7e6c+bbNdKVlhlZsxHAznjEkIyjcuWOq2aZHhG4Ar2xnZIIbqRD50FzbRy3trdRttSSOWAPxLMD/u5PO35ArkxBlK5HRaEqQ4i7r+2qnw76Q+O2xJtrqGEnIMYrPCoSwGeQYpbDMDM7+us9wR6BXOleLR2VqQJJC809w4LLBCrAzTuBtbawAXZrO6LmM8xutBNEn0PZca0jRYZ4ldsJwubLj76+1WSG4lgPnwYdbueNaSTUJZIwuZIBuPoLaeWuEY/P5UeKEXdq0Ed1IQipPyiKYJI5yWOOQK3nEgayRjpzFFqDNWvCtiGC6BznDMFsrTFMWjWM3eJYqvLUgkdVmRILY5Q8eEcNrIFCf3asZWDal19D3hXxPGdJobe/nDW3Jrp+TxRWtpGCkWaeZbRoDq9GedYH6U3BLiVrpBf3McF7d2l3dS3dvdQRyXSlbtzNxLGINxbxsxjCtlmqKRmDWh+i/axaG4rbXukx5HcXf8AsVpZzZRzQQ3LDjMTvo286zsw8aQpxuqX42V8gkTNRFjPpY4fLdabYtDapNNM0sGpFErzO+rhtq7aqICzZKpJyGwAnoqD9EP/AK4YN/3iX/0dzXvPCNodc4Rp3eaRXiCPBooJLw3heNVcnBTYx2yAsGa7kuysaxgZnWDbq8/+h/wS4ocbwfFjbsuGBpZ+Vs8AUx8nuYQQmvxpJlyXILntzOQzNEVb9Pn/AK2zf9zs/wDlGvAa7B+mNwJ4vjOkLXuF2zXNs9rboJEkt0KyRKyOjJLIrgjYc8tUhhtzBA5BniKMysMmUlWHUVORHtFESGUowZCysNzKSpHcRtFX0OkKzAJfosy7hKuSSL3EZBujZs9edZ6lWa8tuWcSC194t0K0M2jomBewdZl6Y2ySRe8HIHp27PVnVDNGUJVwysN6sCpHeDtFIZShDIWVhuIJUjuI2ir6HSITKExBBMvRIuSSL3EZBujZs9edX+R2R5e3NZf7Yf8AIaO6Hks9StDNo6JgXsHWZd5iOSSKPWDkG6duz1Z1QzRFGKuGVhvVgVI7wdoqjmFt61hxmvuNuhX4pSlVWiUpSiJSlKIlTcLxSS2OcLMvWu9W71Ow9++oVKkEi0KrmhwkRMLScstrzZcLyaXtYhmjE9Lp0bT/ADqHimj0kK666ssW8Sx+eMusgbV2fV66p6m4Xikls2cLMvWu9W71Ow9++tKYd2hxCwqnM+mbMDdwN45jJQqVpOW217+MLyeY/lo9qMet06Nv9ah4po9JCuuurLFvEsfnjLrIG1dn1euoMMymLQrN2gTov+U5+huPnkqelWV5gU8TarRyn1oDICOsFcx9W+vj5Lm7OfwP7qqWOHcriKwiYIUOlTPJc3Zz+B/dTyXN2c/gf3UonBTWNxCh0qZ5Lm7OfwP7qeS5uzn8D+6lE4JWNxCh0qZ5Lm7OfwP7qeS5uzn8D+6lE4JWNxC+FrcNEweJnRwc1dCUZT1hl2g91bmx4QhcoIcbhjvYgMhMAI7iMHpVxkG7vNJ6Saxvkubs5/A/up5Lm7OfwP7qUTglY3ELaXOgMd6plwGZLlQM2tZSsNxH6smyVxv2+aNmzWrDX1o8DmOZXjcb0cFGHeG21JtrK4iYPEl0jg5q6LIjKesMBmD3VurHS2W5QQ45ayXsQ2LMEaKeIHeVdQNbo2ZrnltJpROCVjcQvNaV6FjPB8ksZmwmUSJvNtc5Wk6eocbqrJl17M8tmtWKfCJgSDFPmDkfMfo+qlE4JWNxChUqZ5Km7OfwP7qeSpuzn8D+6lE4JWNxCh0qZ5Km7OfwP7qeSpuzn8D+6lE4JWNxCh0qZ5Km7OfwP7qeSpuzn8D+6lE4JWNxCh1ZYVj1xa5cRLMijcgZtXacyNTPV2kno6a+Pkqbs5/A/up5Km7OfwP7qAOFygvYRIkLSppKt2Mrl7m3k7aJ5NUn9NM9n294qLidrdxLrxyzzRbxJFJI4y62AOY+0euqTyVN2c/gf3VOweK7hkUWyXQZmChFSRtdmIAXUyOsxOwDLPqrWdLtDiFz0aH03DwJmOHeOYyUPyzP2tz/AJknvqJxza2vm2vra2tmc9bPPWz362e3OvRsZ0QueLEmLWGL2GZAF21rdQxMzZZa3GIBmT3k9YrJ4tovNBtUGVOhowWP/mT8IH2j11UwjKbbQrt2ls6L/lOfobvXJUkjliSxJJ3k7Se8mv29wzausznV/BzJOrnlnq57tw3dQr7+Spuzn8D+6nkqbs5/A/uqlE4LasbiFGmlLnNyzHrYlj7TX945tXVzbVzz1czlnt25bs9p2+s1I8lTdnP4H91PJU3Zz+B/dSicErG4hR5pmc5uWY9bEt/GizMFKgsFO9cyAcjmMxuO2pHkqbs5/A/up5Km7OfwP7qUTglY3EKHV7gWmV9YJxdjdYlbR7SY7ee4tlJO0nVicLmarvJU3Zz+B/dTyVN2c/gf3UonBKxuIVlBpXPrObljdBznJygmZnO7Mu+bFsusnur78htr38WbiJj+Rl/AJ6kfo2/0FU3kqbs5/A/up5Km7OfwP7q0DnSkRMLBzIc6THUTl6i4+eav4NIsUwheJgucVtItuSQXFxBGcySdUROEOe0/XWavLl5naSZnkkYlnkcl2djtLMzZlmPWav8AC726hXUeOaaHcYpEdhl1AkZj+HqqU+j8d2M4FntpN5ilV9Qn9GTLYP8A5kKmqpdnmq/FUPqS8RaOo5jNZy8xWaaOOKaSeSKMZRRu7ukQyyyjRiVQZeiBUd52YBWLFRuUkkDuB2Cp13gc8TFWjlJ61UyA9zLmK+Pkqbs5/A/urIscO5dAisNoIUeCZkOaFlPWpKn2ivwTnUvyVN2c/gf3U8lTdnP4H91KJwU1jcQodKmeSpuzn8D+6nkqbs5/A/upROCVjcQodKmeSpuzn8D+6nkqbs5/A/upROCVjcQo0MpQhkLKw3MpKkdxG0VfQ6RCYBL9FmXcJBkki9xGQPRs2evOqryVN2c/gf3U8lTdnP4H91WaXtuWcQQn3kT8ZFWs2jomUvh7rMvTG2SSL3g5Bunbs9WdUM0RQlXDKw3qwKkd4O0VMhw+dCGRLlWG5lWRSO4gZir6G9lmUJf280y9EgRkkXuIADdGzZu251eiHd0jy9uayrHQ+8OHAO6HkslStPf6JsV17Qs69nIphkX1ZPkG79nqzqk8lTdnceB/dVHQnNvC2h7RDeLD6FQ6VM8lTdnP4H91PJU3Zz+B/dVaJwV6xuIUOlTPJU3Zz+B/dTyVN2c/gf3UonBKxuIUOlTPJU3Zz+B/dTyVN2c/gf3UonBKxuIUOpuF4pJbNnCzL1rvVu9TsPfvr+eSpuzn8D+6nkqbs5/A/uqQHC0TVXOhuEiQQrLD9LLiJs2bjF6Vfb7CNoP2eo1dDFhef7iaW2mP5KQ5xsepH3jafctYelXbHcLDaFi/Y4bjMCRy6XLRYre3tq2UzzL1NnmrdzDYe7fULnFcdrL7a/WFaQSwLqHVlh3GKQa65dQz2rs+r1VN5JbXn+4bk0x/JSbUYnoR+jMnL6ti1aZd2XHwJVKLWfUYJYgWcReOYzUDnFcdrL7ac4rjtZfbUfFcKltTlMpXqberdzDYe7fUKsy94MiSt2woLhMAEeAW64P7O7xd71Y53j5Nhl9iTkgvrJh1uZzGMssmfIKG6M+ms1ziuO1l9tdL/Rw4LJ7HRjSfGcQjkhM+j+J2tkkgKM1u1lNNNcFG2rG7RwhGO0hZCPNZS3K/Et1N7DUVjsSrVEPdGisecVx2svtpziuO1l9tVkUZchVBZicgACST1ADaTVrc6L3kULTyW98kC6pedoZkjTXdY11pCuoubsqjM7SwHTSsdiUqIe6NF+ecVx2svtpziuO1l9tf3B9Gbu8jeWzt764iQkSSwwzTpGQAxDvGpVCFIO0jYRVTSsdiUqIe6NFac4bjtJfbWp0Z4QAi8ViMSzxkZcdF/cTpv2hlIVyNm/VPWTWTwDArjEJeKsIbq6ly1uKto5bl9UEAtqRKzauZG3LpqbpHoZf4agfEbTErRCdVXure4tVY7dgaZFBOw7B1UrHYlKiHujRbe5wOS9Uy4DdtcqBm1rKyxXEf1EAOPX5vq1qwt9i93A5jma4jkH4SOCjDvVhmKqLadonV4mdHBzV0JRlPWrDaD3V6FgWmb4iBb4pbNiSjYskSHlMQPSrRjztw2ebnltJpWOxKVEPdGix3OK47WX205xXHay+2t3pHwSyaiy4dykqwZltryOSxnIj1Nfi+OVFmC665lcgNZd+YrJQ6CYi6F47PFXQEgulvcuoK7Dm6oV2d9Kx2JSoh7o0UHnFcdrL7ac4rjtZfbVW6kEg5gg5EHYQRsII6DX8pWOxKVEPdGitecVx2svtpziuO1l9tT8C0AxK/iE1jZYvcwndLb211cIcjlseJCp2g9NRLrRS8ine3ltr9LlInnkt3hnSWOGJWeSZ4mXXWFUVmLkZAAknZSsdiUqIe6NF8+cVx2svtrrDQZG0S0Dl0g/Dxm9URWk0uUnJIrmdoIjCGzRX4gSXBOXnHi1bMJlXHiITuBPdtrsvhzGfBZgX/AN1/+muKgvce9SILAZho0WW+j1w9WGHYJi8GkhxC8vriSVw0uvecshktY4ktzLIx4tllWViXyH98CCxBA5yh0kuFIPGOcuhsmB7wd9VToRvBHfsr+KM9g30D3C4o6Ex14BWzttKluAFnaW3k3CWIkof2kOeX294r4Yny2FddJXmi3iSI64y62A2j7R66LwY4uY+MGHY4Y8tbjBZ3hXV9LW4vV1fXnVBYX8tm54svGwOTIetTkQ6NszG7aMxW1eT2p8FynYw21gHgRMdRzGS+/OK47WX205xXHay+2rS3uIMRYJLG8U53SQKzhz+lGoJP1An1ivzpDoHeWUfHSQ3RtswBccVKiZsCQCWUAMQDszO41Bp3gzCs2qnRcwA5gS4G71yVbziuO1l9tOcVx2svtr+22jN3LbNdR2989ooYtdLDM8KhCQxaYLxYAIOeZ2ZGq60tnlYLErux3KgLk9wXM1nWOxK3qIe6NFYc4rjtZfbTnFcdrL7aYpo3dWsQmure9hhLiNZZYpoUZ2VnCB3UKXKqx1c88lbqNfXA9E72+QyWNtiFzGDkz28E9wqkbSC0akA+o0rHYlKiHujRfLnFcdrL7ac4rjtZfbVfd2zwu0cyvHIpKsjgoysN4ZWyKn1GrHAdGLvEAxsLe+uguxjbwzXOqcs8mMStqnLrpWOxKVEPdGi/nOK47WX205xXHay+2oWI2MltI0VwksUqnJ4pVaJ0JAYBkcBlORB2jpFR6VjsSlRD3Ror/DdLZ4mzduNXpV/9GG0H2j1VcjExefi08tvL2MpzVj1I+07/AOlZDDLCS6lSG2SWaZzkkUStM7nInJEQFmOQJyA6DV7jXB7idlE017Y4xbwr+FLPa3Vui/tPIgUfWau2O4WG0LF+xwyZtEj4WaXeua+WKX17bNlM0y9R3q37LDYah84rjtZfbX7wvSKSFdR9WaHcYpPPGXUCdo/h6qsLfC4MRZVsteO5chUtiGk4x2OSpFqgsXZiAFGZPQtWmXdlx8CqUWs+oweIExx7xzGatLCwu5cFusV49xHBe21lxWRJke6huJtYNuAQRDMZbdcbstuY5xXHay+2umOE7g1l0a4N4Ibwat7cYxBeXMYIfimlt54ooMxvZIY49YbQHaQAkZE8qtGRtIYDuNZVjsSumoh7o0Wo0Kxsy4hZJiU0yWTXdsl24JUpavcRrcMCoJBERc5jbsro/wCmla4Nh9hhz6Oy2kd68qgx2cwnEtgYJW5RMFZwTxqxBZW2vrS7W1Tq8l2lq8zBYleRzuVAXJ7guZqfimjd1aRLLdW97BEzBFlmimhRnKswRXdQpcqrHIHPJT1UrHYlKiHujROcVx2svtpziuO1l9tVVS8Iwya8lWG0jnnmb8GKFHndst+qkYLH6hSsdiUqIe6NFK5xXHay+2nOK47WX21YY5wf4lYxGa+ssXtoRvluLW6t0He8iBR7azVKx2JSoh7o0VrziuO1l9tWuF6YuBqXWs69oh4uQevZkrdHV686z+HYdLctq28c0zdKxI8pGe7YgJr7Y1gdxZFBew3VuXBZBPHJb66g5Fk4wDWUHZmKlsZ471R+ywnCRaOFi003HzKXw+4eZQMzGxCSL3ggA9O3Z6s6oJsdukJV5JlYb1bzSO8EZipWH6H4i0S3Nva4qYctdbiOC5KauWessqpq6uXSDSHSITAJfosyjYJBkki9xGQPRs2evOtKyl3kHl7c1jUVf/1Dh4AO6HkofOK47WX205xXHay+2pk2jyyqXsHWZRtMbZJIo9YOQPTt2erOpGF8HGKXUYktbDGZ4j+DJDaXcyN3OkZU/Uao4xG3krSG2A+4CfhI+Sq+cVx2svtpziuO1l9tJtGrtJZ4Xt71ZoImnuImhmV7eGPVLzToV1oYV11zdwANZdu0VVVWsdiVrUQ90aK15xXHay+2nOK47WX21VUpWOxKVEPdGiUpSqLVKUpRFb4VpBLANQ6ssW4xS+euXUM9q7Ojd6q0uhuOCzuku8Gl5BfKGCCRYp0/vFKME45WQE62QJGY6AKwdK0EQykbQsHbOJ0m/KcvUXHzzX+gnAjp5il1oVpFfYnO8mI2vlU29wUgXixbYJbXUBVY0EbhZnZvOU7SQd2VcrS/ST0iYFWv5iCCCOJstoIyP5Hqr3f6Ks5/+m+lBbz9Xy3krZsNVdHrI6m3/B6vWa5Thw+3xBlW1LQXDEKsDZusjsclWIjbrMxAAG07gtAwOu5o6KWAUxxF3VRdBtLbrBLtLzC5Db3SK6rKFilyWVDG41ZlZDmpI2iu6tF8Vk0y4PHlx+UZGQSX1yipCWs8IxeG8uXVIwI1nNpbuo1VA1sjlXBGNYNNZuUuUeNgxU5gjzlJBX1MCDmpyIy2gV2TwXSlOCTECuw8TiS/VJdFGH1qxH11QgiwrVrg4TBmF5vwIfSJWyx43GLi4iwhbSW1ssOtM2t8N1pYWhZLcFVkkESSxtPkZHMxJ6h5Bw16SW+MY5iF9h0bW9rPOZY4mCo21EEkjqhKrJLIHlIBORkO076x1KhWXXf0ptArnQ/ArBcGuzb4aZYrS4trZXtJru8ltp55L28ukkL3uuYXAiYKkSiFVUgEjzj6LPCpc2uL2uHYg8l7hN/KlhcWV0TdxZ3jcRHIkcxKJlK6a4yydNYEEhStxDwRXjYTb3+neJXWHYWgRbOzuGuMRuSHUBYre0dtW2Zo1XJQGZVUl0RVJqo4N8S0ehx3Co8MtMfuXbErFY7q+u7a14mU3luqSC2tLd1kQN52o0ozyyz27CKt+l9wbQ6M4+0FiCtncQR3sEZzPECWSaGSAO211WWF2HSEkjBzI1jmuC3hgxTRlWTCJxDC8yzSwtFbzrMyBVycyoZFUoNU6jKctxB217Z/aSf9M4Z/4cf/AFk9cq0Rf6D/AE1rqCxt8Kxi5WOaa25VDYWkmbRyX+IizkjnuAP95a28FpO5jOx5GgU+aWB5f0c+kPiKzI+IzXk5Vw6zRSNbyRlW1wojQrbyQZgAwlQjDMMCNle5f2jrHybgQ6OOuCR6xb24B+0+2uJqIu1Ppk8HkeOYdhWPYVFGt3cRwmYLqwtdQ3VlyuFnB2PcxqhUEnWKtkSQiAeP/Q64OYsX0m4jF4i0drazXzWkylVmeGa2gijmRh50WvOrlTsbi8iCpIPsX0qrhouDzRZ4mdHV8FKuhKMp5vXu1WXaD3V4ZwScPFxg17Bc3SJdNGGj43JY5jBJlxkTMNjo2SnI5eciMSSooix3CZpxiWJX05xeW7EyTOjWpaSKO0aJyht4bfPVgSMrq6oGeanMk5k+v/Re4TbqaHGsLv5JLmJ8CxWS1ecmeS2eGzLyQwyvm6WskaazRZ6utChABLE++RaU6Jabrxt9FatclVEnHRyW1woXYA9xbasvFDLIFnAy3ZbqwfCr9FeyubKa+0KmdpY1kBsxMLuOcKhEtvDMCZYrkozDUkZwxIU6mZNEXK/B3whX+j0ksmDzvavKixysqQy66I2uoymRwMj0jKu1+FLhHxCz4PcMxW2ndMRlTDzLchIWLm4VzLmjIYhrHLco3bK/z/rtThq/7KcH/wCHhX/6Hoi5f4ROFPE9IY4Y8YuHukiZniVkgi1GdQrHOFEJzAG/Ove8Q0ej4NNG7W+aOB9Kb86sEsypMcKjMQeY28bgryiKN0Rn2/3k4HnIuq3PnA1arPpBgsUoDRvi2HRup3FJL+BGBz6CpNdMf2mDHjsCHQI8RI7y+Hg/YFoi5fl0+xJrjlLXmK8pzz4/lFwJAfU4fWHcDXReieDJwn4BePOsS6UYdqat5GqwnEoZI5DBHdhQqPI/Eyx6+9GSJswrsjcoV1r/AGaznyji4/wmztye9bhwv2M1EXOPB9pxfaOXUlxhMrWtyYmtpG4uGY8W0sUrxlLhHUf3kMZ3Z+bl0kHunTTHPK2g2F4tjJRkhW3xK9jUiAXjwW91DFbDIEIt1evbRtqjYsr6uRC5cW/SKsUttJ8bjh/A8pXTjdkpmmaZkGWwKrOVA6lrprhJlK8EdiBuaHD1b1gXyvl4lB+qpBlcoIBEisp9H36TUNreXsukvKzxsMMdtxGclvapAZda3gtCwjtYXVolGoPyK6x2k15TaaaYhgOIYhfaMstnbXU8zqkCW10sVs1zJNBAVljdEWNWC+aMtmWeVeUVMwzE5LZs4WZesbwe9TsNXDmntclk5j2j5Dr1vHPwXfvDdew4toPhGLY4FkjiWwxa4tUziF7cy4bcW0NqpXNooJL27iLkZlYllyyORHF2M8LWK3MwkW7vbZVyEFvZSSYdb2saDVSK1t7ZkihjRQANUZ7BmSczXV/Dg6S8GeC8exiV4cH1mUbATalhmPQ1gNnqFcY4po7JCuumrNDvEsfnDLrIG0faPXUUCRMKTGaCGusJ08JrtHSvA7LTDQ3CtIMbGVxaQNNe3MSpFNfQ4e9xbXNpmuqoe5nhGodgjeQldUMwblHSPhaxK6kHJ57mwtk822sbCSWxt7WNWZkSKKFlBcZ7ZXzkY7WY10irZcC+zMbx9R0wyI7sq4yqi1XaOL240z4NTiGJ6kmKWCXDR3zAcYwsJ83V2GTMJbTJWBJBkCyEEgVxdXbPAV/2V4z/AMDGP+RXE1EXXUnBvcYNwfw4ngl0lnPLax3+IyxqYbi8juZYjDax3yycZbwQoygwIMpX18yNbVrwngp4YcQwC/iuYp7qaHjAbm0lkkljuoiRxiOkhK8aVz1ZMtZSc+sHacGPBRieK4I0+K38uEaMR/3oN3JPJC5MpIkt7DjFjfWmbIMShd3Gprk1nsYudGLF9Wzh0ixUg6rSz3FthEL5Z/3kKR28txqnZsk1T3UReifTr4MLXCp7LFMJRIba94wTQxgJGk6LHKkkSDYiyxuxKKAAYif8VeI8HXCPiGjxmODzm2M3F8awjt5i3E8bqZGdHKAca/4OWee3PIZdYf2gTA6O4EVBVeUIVUnW1QcPbIFshrEDZnkM64ioi/0G4YeEu/w3QTBcUtpmF9MmFGebViJl5VYPLN5pQxqWcA7F2ZdWyuYdLuF/EdI7KWyub6bipeL4y0nS0RJDFLHNGFliiVxlKiNvzOW7KvbvpF/9l+j/APwsE/8AbJK4xsrKSfXEKSyFI3lcRq0mpFENaSR9UHViUbSx2Ab6u18rCJrKJCLrQSDy0u9c1stCtNcV0TlmbC5XtGmCJI/F29wsqxFmQAzxumwsx2ZHbtrqH6auNPiWg2j15MFWW5uMNupFXYokusEvJ3C57dUM5y9VceYZpDJCuo+rLDlkYpPPGXUCdo/h6q7A+k1aC64PtFVQpFrLg7Rqx2Z837krHrd2zP1VJaD2VVr3tBrBd3i7S8c/FckcG2iU2PYnZ4dZ5cdcTCMMd0aAGSaZhmM0ihWSQgbSEIGZyFe9fSM0mTQwpo3ooWtNSCGTFMRjPF3l5PKvHJFJOgDxxiJkkIRgDx2oAqqweV/Z9YE0Okt2bkAOmFTmIHI5lr2xjaRGGzzVJU/8Ubq8x+l45bTDGS2/lEQ+pbO2Vf8A8oFUIIsK2a4OExaFmtCuFHE8IuVuLO6vAwbWeKSSSeGcHMMlxC7FJUYEjzhmM8wQQCPYvpEcHNrf4DY6WYBEltBcLH5RsoshFbzySG2eWEbAqLeKYGVQAS0TBVzeua67w4BLdbjgsvo5iAnIsdyZtoTU5XKr/wDkk87vFQpXMfANwwYngE9taYdMEspMQilntmjt5VmaZreCXN3QzJrRRovmOuWWYyOddO/TmvIcKkscTkW3mvxDJaYXDMqzxwzNJx1ziMkDji5mto+JWNHDJxtyjsrcWAeJNDf+kLL/AL3b/wDPjrq7+0wb++wIdHF4icu98Pz/AID2URc94Jwz4xa38d9y7EpZVlEjJNPNLFKA2s0UkJbizARmNQAADLVyyGXuf9oXobbwyYZi1miRPdiWO61QE42RUhmglZQMjMUaVWfedWPqrkyu2v7Qf/oHAf8Aj/8A8EUReS/Qh0JixTFL26njjuWsbJrm2tX2rNeuxW311Oxo11X2H/E0Z6NvnuP8JF5iM7nHnu7mUSMS7M8MkDhjmqRqVjiCts4tFQDLLLor+cBPCfPoniiX1qqyoUaG4tmPFi4t3Ksya+RMcgZUZXAORUZhgWU9Yz8IGhGlp4zFo7a3u5Mmka5imw+YsoC5y3tmRG+QAALzHMADLoFmuLblSJDa8SPQrzzgd06mvsB0lw6WeS+HkK7ntZJzxlzAkI4p7V5H/vHty06sinzVKyZfhVyvNEUJVwysN6sCpHeDtFdd8MH0Z7ZcOlxfQi5kmiSOSV7aOVbxZoIwDJyO5hOszoEYmFzIXOYDAhUbmKHSITKEv0WZdwkGSSKN+wjLPo2bPXnV/ldl5LL/AGQ/5DR3Q8lnqVoZtHRMpfD3EyjaY2ySRejaDkG6duz1Z1QzRFGKuCrDYVIII7wao5hberw4zX3X6FXnOhuxsP8ALPxU50N2Nh/ln4qoKVaufiqfCwsFf86G7Gw/yz8VOdDdjYf5Z+KqClK5+KfCwsFf86G7Gw/yz8VOdDdjYf5Z+KqClK5+KfCwsF239Ea+8qaFaTWkC2/KycQAt0GQPLcFht7cumZJWSSGVfXqEdFc48C2JSX2P4PBFDZlmxKyOaRnNEjuopZJB5xyCRozk9AUmqHgq4R73Rm85VhMgjcrqSxuOMinjB1tSaM/hDPcwIZczkRma0+O8NTy8pksLHA8NvbmOSO5v7OOdZis4KzC246V47PjVZ1Zo11yGPnDfUVjrVcwGGVlyrOEDT0z4piLwrbSW0l/eSocmPGRS3c0iNnrau1WBzy6a6o4O5xPwXXzxJGitHfsEcDU2Xh1i2rkDHmDmfUa4y0D0ofB7xLqGOyndVkURXcS3cR41ChLRMQCwBzB6DlXsFv9LXGY4hFEmDJCFKiFLYIgU55qED6oU5nZ6zUmK4yBVBszGklokSCNV5bieKzWxylt7EDoYRllbuYNke7fW3+jViUN9pRhEF/FYLA12Dnqauc0UUktsmbMQS1ykK6vSSB01S6bcMdzi4iW6t8GjRC5K2tslpxokCAiUqTrAauzqzO+s7BBb3LK9o7WlwCGVGJC66nMGOQbUbWyyO/ZsArWkXdk8CucQ2wz87eIu4iZI5hdE/2guPzwY7ZxSRxPbLh6vbmVWkXjJbmcXBQ5gCQ8XCGG/JIs94z8o4Bb177HsPzjsYoLe4ixC7uChRbWzw2RLy4nkkzKxKEj1QzbNd413sAbjHuHDEZrSOx0mtcLxeBMxHNeRyrOM11c4ryzliZXKgZvlrsANYk7awek/CLJdWjWNlBh+GWLMjTW9ikqtdmFi8HLbm4klursRsSyo8moGyYKGAIypvb8q6RChP8AnFq6T+nvK647gyJFDKJrUQrxiGQs/LmUxocxt/vE2fpCvBeG7EYrLHsWtbGDD1tob65t40EexFhkaIoNv+FlYfVVxF9JHFOQ21tOuE3MtsMrPErq2W7vLRlCqkkM0jGMXCqqDjTGXOopYsdtZvg34XLrAhNyeLCriWSflLXF7At7MsuQzKTO2suZGsenWJOdQIjhcVLoDHGZC6k/tCr021hgjakT/wB9cqRKpcKeT2+7aMm2H2GuOOdDdjYf5Z+KvXMV+lfi92nF3ceB3EeYbi57UTLrAEA6ruRnkTt9dYPQ7hVnwu7v7qC3wWSS7m490uLWO4jt242eXVtEY5W8ecxGqOhYx/hoIjgJAo+AxxmQuofpU35g4P8ARpykJbXwcGORSVUtgF7mAuYKkbvVXl30Obby9iGMWDpZx8do7fwq6xnJXlvMMRGcZnWRXKtl+iKrr/6WmM3EZjuEwWaI5ZxS2olQ5HMZozkHKsPo7ww3mHYliWI2SWVvdXllJZHiI+Tx2qyvaMZbWJG1Y5xyZcicwGZmyNA9wElJgsJmRaqjF8XucNu5ILi3tbe6glaN14tkeKWMlGAZX2Hf5ynIg5gkGvePoQ6eXdxpDJHKf9lNjcS3bl3VIobYBkmlEhZfNmZFDZrqiWTbkxB8hHCy11FHHjtnhWLtGixpd3Iura74uNQsccl5ZTQyzxqBs43Xbb+FXxx7hbuZbKWww6HDcJsZRq3EOHRvE94qk6i3dzO8t1OqgsNXjAp12zU51JiuNhKhuzw2mYC0+k9mmJT3V1gBwy5R555uSyR8RPGsszuBqsyhhtO0hd2zWr3rhomeDgxwpnjQOqYXrQyIQqEqwyKHIrkSAM+sVyRwYafTaP3Ek9pFh08jxiP/AGyFbsR5Or68QYjUkzGWsOgmvZsI+l7iTAxYnFhs1swCskcIGSjo4uRjHIuweadXvqDEcVIgME5C9eJYfpnJbyxywxWKSxyJLG4jOaSQusiMPO3hlB+qus/pJ4wul2iuH4/hCQXDWjSLfW7Lx7Wa3McHKldRk44mWKE55bYpOM/B214dwnwHSqRLvCjhTcXFxZtLaGPC5QvGO+tJGDqSyZsw18kGQAGtlnWH0D03xHRa7eTD5JrWUgJPBIuaTIpOSXEEoKOuTNkSNZdZtUgnOhiOnNQIDACALCq7nQ3Y2H+WfirrX6KGIro1o/iukeNJDbQTCKKziROJkuxbCdgItYnXM80gRNgy4iRz5nnDwJuGCAvxxwbRI3WeevxF2IDt3myFxyct9WXq6Ky3CVwj3+kUyyYrM0qoCsEChYYLZCfwIIIwI0yUKutkXYImszZChiOIkSjdnhtMwLV88a05lvLie4nisGlmnmuJW4s+dLcSvNIdrZ7XY11xwjXZTgqw+XUiP93h7cWVJQBrwZHVz2LtGW3pFcicG+mkmBXL3FvFh9w7QtBqXsK3kahpYZddEYgLMDEAG6Azjpr1iX6W2NNFxTLgxh1QnEm2BTUAACahfV1AANm7ZQxHFSIDBOQvXj/OhuxsP8s/FTnQ3Y2H+WfiqdwlaeS49LDJcQ4ZbNGjIBYwJZK4Z9ctIqEh3z2Z9Vfrgz0+lwCSaS2hwy4aRUQ8tgS9EeoxYNEHI1HJO09Oyprn4qnwsLBdbcOl2Y+DLBZAkTf3ODMUZSUUPaHLzc9i7Rlt6RXH1vpjJHnxcdmme/VRlz78n2161d/S0xmaIxTJgskJAUwvbB0KrkQpRn1SoyGz1CvP7HhTnhxefFFt8EM00SwtbNaxtaxhUt49eG2J1Y5coFOsDveQ/wCKjYrm3FWfs8N/aE11NaXQfgjWSdY0DHWYFCyDW0szDlM8yp2Pnn0551ylic80K66Q4dND0SxRlxl+kA2a/aPXXo//ANrXGeK4nUwbidTi+J5MOL4vLV1NTX1dTLZq5ZVhJ+Fy5bFocTWHCEljiMXJY7dY7SUGOePXmtVbUklAmJDHpSM/4asyNIEHxWUXZQ4gjuEpG6XnNdR8Cl2X4MMYlKRL/c4wwRVyRgsJz83PapIOe3oNcj6PaQJNdW8d1HYx27zwpNIEK6kLyosr5ltmSFjn6q9lwL6WWIiIQEYVbJkVUR2oMIDE5gxBxqqSTmBsOZ2ivMuF/G7rGXiuJocLVI0ZVkw2BLVXV219eZEJzf8AS3DrqRTlMHRQ6qJDXiUrBP07vVdP/wBoRiMthYYJDaxwnD+NuA6FCYklt4LdLNAFIVcoWudVepWy/BrknRy8nxO6gs7O3spbieRYYo1jPnPIdUZ+dkqjeWOwAEnYDWs0W4f8QtcOGGXyYfi2HBVRbbE42ueKjjOarFKjpKupsCFi3F5KFyCgVTXvCi0UE0GDWmF4SkqtHNNai4nupIZBlJByy9lmmit32ZpCY9YDIkjMHMRHASBWz4DHGZC6n+nTiKJo5gc9rya6tzcxiOYjjY3STD5WikQg5ZOikjbuzrnHhouIsOmwyO1gsEZ8Cwi5uU4s5i7u7NZ5mcawKyNrKxB6xXz0M4e7/DsK8lSxYTiNgDnFDidvy8QecXAjBdVKq5LKJA+rnsyGyvOtJ8dnxO6mu7+R57mZzJLK2QLNkAAFUBURVAVUUBVVVUAAAUERwuUugscZkLs/hdxEXXBdhN1arBKkMeF8apXXRDArYbKNUHZqXLBN+w7K8G4C8dJTSCeWO0jhi0bxNGkCFFMt+IbG3hZics5ZplATe2Ry3VQcFXDRfaPW89nCLS7w6cMJsPvozcwNxgCuyhWV0ZlGRAbVOwlSQKgaZcJD3to1lY22HYXYvKs81vZLKGu5YgeKN3PO8k00cReQpFrLGpcnVJAYBEcBJDBYTMhUnOhuxsP8s/FXXP0oL0w8H2i0hjhJywcGORCVQtgF0SAuYKkZEAdAzrlngx4RJtH2na1hwu4aURZm9t0veL4kyEGHXI4snXOeW/VXqr0vEPpZ4xcRmK5TBJojlnFLaiVDluzRnIOVDEce9QNnhiYAvVR9HrhlOBY3a3F2sS2RDW11xSvmlvPq5uqhmzEcixSFVUkiMgbSK9O+mHo2IsWGI23IpLbEIoJ4Ll1E0MsqW8cJiWeM5azRxxSqTnriQ6ueq2ryzpBiZvbq4uXWGNpppJmjhUQxo00jSFIoxsSIE5BegZVtNAuF++wi0ew/2O+wxiWOGYjEL62Dl+M141JWSFg+b/3bqNZi2WttqzYzg6Zt7lR+ysLKDbLZ8VWXd7cRyLG1taF2YJGFiaTjHY6qrHqsddycslG3aNldWcK+lQ0L0Gs8GlEAxe9tn420ABFvFezvPemZUJAQCR7ZfO89tcrrBHy8Kwrh8fDyGwjDsAspRnlcKlzeSwkjVJtTeTSR2+wkZBCNu7LMHGaQXi47cSXU003LpW1pOVMZeNbIKAsh3AABQo2AKoCgACrEl9x1v/f2Sza1sHtNymLpazHlmv5olpC0l9ZpxNl511bAasZzzM8YGr534WddQf2jWIm2nwQhIJNaLEBnKpfLVewJC7RlvGf1Vz/oBwmX+icZht7fCWblDXCXNzbJdSq7RxRgQ3GsCsYEQYKNoLOemtPi30q8WvFC3kWBXKA6wSe0WdQd2YDuQD6xVHPeDbeFsyFCLfluK8kfSYkEcTYbsv8Adnp/81dlfT2vTb4JgbakLnj9UiVS4UmxG4ZjJth+2uWeD/hZuMEe5e1gwaR55+UFrm2S5MLguQtsSRxMebfgjqHVW6xT6WOMXScXdx4JPHmDxc1qJlzGeR1XcjPadvrqDEcTOakQGAEAXq++ihhTaRWWk9kkdmssmG2/EDV1Ee5huJLi2EpJP93yiKPM9AzNeFXuPSQSPFNb2ccqO0ckbwvG8ckbFHR0ZgyOrAgqRmCCKvNGuFy8w1saewFtbS4mpWaSBTDyVXnkmcWSq2rACJHQHaUBBUhgGEm44WOXKnl2xwjFJ1UKb6UXdjdzairHHyq4w+eHlRWNQutKrOQq5sSM6mtdioOzQz3L2L6COmkseIYpxwhhwqLDJry8ZFZIo3gkg4uVySUEvFceMthZVffqbOcjpOf8MNgB0Diych0DPW25CtBpbwr3N7YnD7WLD8Mw5mV5bPDo2tluXTLUe7lkeS4uWXIbJJCuaqcs1XLAVFY6c5qTs8MgAi5aGPSplIKxWII3ERsCO4hsxUu30w12HK4oHXcCq+cv1SFgR6sxWTpVhHeO9Zu2OEe5aXylb3my7XiZT+Xi3E9bp/rt7xUTEtHJI114tWeLoki87Z+ko2j7R66pal4biMls2tCzKekDcf2lOw/XSmHdocRf7qalzPpngbR1HlkolK0vlK3vNl2vEy9vCNhP6abfVt2/VUTEtHJIl14tWeLeJIvO2fpKNo+0DrqDDN7bQpbtAnJ4onO7gbvXJUtKUrNdCUpSiJSlKIlKUoit8L0glgGodWWLcYpBrjLZsBO1dn1eqpvI7a9/F25NL2UhzRifQfo2n3LWbpWgiGUjaFg7ZxOk35Tl6i4+eam4phUtscplZept6tv3MNh3bt9Qqt8L0glgXUbVli3GKXz1y6hntXZ9XqqbyO2vfxduTzH8lJtRj1I/RtP/APmpoB3ZPAqta9n1BZiLuIvHMZrN0qbimFyWzZTKV6m3q3cw2Hu31CrMgiwrdrg4TBmEpSlQrJSlKIlKUoi+ttcNEweJnRwc1dCUZT1hl2g91bmy4QhcoIcciS8iGxZlyinj9auuQfo2Zrn0k1gaURegXWgMd6rS4DMlyo2tayEQ3EY3ZZNkHGee3zd2zWrDX1o8DtHMrxyDYyOCjDp2htor82tw0Th4mdHBzV0JRlO7MMu0Hurc2PCCLlBDjkMd5EBkJwBHcRg9KuMg3Rs83PpJoiwNK9AutAY71DLgMyXSgZtbSlYbiP6m1Vcb9vmjZs1qw17aPA5SZXjcb0cFGHeG2iiL4UpSiJSlKIlKUoiVMwvE5LZs4WZesbwe9TsNQ6VIJFoUOaHCREwtLy62vfxleImP5aLapPW6dH/zaKh4no7JCuumrNFvEsXnjL1gbR9o9dU1TMLxOS2bOFmXrG9W71Ow1pTDu0OI/bVz1LmfTNmBtHDvHMZKHStLy+2vfxleTy9tEPNY9bpv/wDm8V9rbQG6uFleyUXSRoJCYSHYoTlmqZ6zsNmarmduzPblV7Q0Up2KW7QJyf8AKc7uBuPnkspSv06lSQ2YIJBB2EEbCCOg1+aouhKUpREpSlESlKURXGF6QywrqPqyw7jFJ54y6gTtGz6vVUzkNtefizcnl7GU5ox/Qfo2/wBBWbpWgiGUjaFzu2cTpMNE5XcRcfPNTcUwuS2bKZWXqO9W7mGw91QquML0ikhXUfVmi3GKXzxl1AnaNn1eqpnIba9/Fm5PMfyMm1CepH6Nv9BU0A7sngVFc5n1BZiLRx7xzGazdKm4phcls2UysvUd6t3MNh7qhVmQRYVu1wcJgzCUpSoVkpSlEW+43A/Rxv8AdvipxuB+jjf7t8VYGlEW+43A/Rxv92+KpWHYrg9u2tD5eQ9ORtiD3qWyP1ivN6VIJFoUOaHCRW/xnEMLupVZVvFBPnF1jhO7/E8TOGz69TMVZz4JhKRiTUxiRCMy0DW86r17c1bL16teW1Lw3EpLZtaFmU9IG0HvU7D9daUw7tDiL/f9tXPUuZ9M8DaOo8sls+NwP0cb/dvipxuB+jjf7t8VU3lK3vPxteJl7eIbCet0/r3iomJaOSRLrxas8XRJF52z9JRtH2j11Bhm9toUt2gTovFE53cDd65LScbgfo43+7fFTjcD9HG/3b4qwNKzXQt9xuB+jjf7t8VONwP0cb/dvirA0oi33G4H6ON/u3xU43A/Rxv92+KsDSiLfcbgfo43+7fFTjcD9HG/3b4qwNKIvULTSLCYozGvlpkIy1JRbTKNmW5mzy9QIFVuEW+FXEzB+VKCfNGstudvoq3GIcyfwdceqsDStBENxtCwds4nSb8py9RcfPNelYnYYPbHKaPHl6m/2Yq3cwbI92+ofG4H6ON/u3xVmML0hlhGo2rLFuMUvnjLqGe1dn1eqpvI7a9/F25PMfyUm1GPUj9G0/ZsWpoB3ZOqrWvZ9QWYi7iLxzGauuNwP0cb/dvipxuB+jjf7t8VY7FMLktmymVl6m3q3cw2Hu31CrMgiwrdrg4TBmFvuNwP0cb/AHb4qcbgfo43+7fFWBpUKy33G4H6ON/u3xU43A/Rxv8AdvirA0oi33G4H6ON/u3xU43A/Rxv92+KsDSiL0K1vMFicPF5eRwc1dGgRlPWrK2YPdWkxPT7CruERXseI3OWYWWVLbjVB9GSJ0I/16c68ZpRF7/gmgGB30DS2RxK6KgFoIJrcSgH0o7hY9Xp3nblszrL4hBo9byNHcRaURSLsaOQWsbL3qzZivL7G8eB1kgaSORdquhMbL3MuRFeh4fwmJeRiDSGCO9iAyW4QLFcR+sMCoY7txTPLaWzrzzAiMM6TnDxkeh5L0DC2Tae8wneJdD9XN/9h4L7cdo16Gkn7p8VOO0a9DST90+KvnfcGsd8jT6Ozx3cYBLW0hEU8e3LLJgoYZ9LBc9mWtnnXneIWUlvI0c6yRSL+EkitGw71bIirw2Nfc92tq5Np/xkXZ5F85G4gzafAiw6r0jjtGvQ0k/dPipx2jXoaSfunxV5fStfh/5O1XJU5nVeocdo16Gkn7p8VOO0a9DST90+KvL6U+H/AJO1SpzOq9Q47Rr0NJP3T4qcdo16Gkn7p8VeX0p8P/J2qVOZ1XqHHaNehpJ+6fFXqP0eZcLNxdDBRiqvxKGXlfE6pXjMhqCIk8Zn19FcvVZ4Lj9xZLMtpJJCJUCSmM6jOitrBdceeq579UjPccxWUfZC9haHG3E2KkXZ6TSATxXQfDnLgUkxF4c7/PJmtCA2YAGV26gx7MgPOBkAA3CvOrvBcJijEhTGXQjMvC1tMo69uYbL15AV5dUzC8TktmzhZl6xvVu9TsNdOywWwWUCSc/3uy5qGwXwwKDp5G7qOYyWy43A/Rxv92+KnG4H6ON/u3xVTcutrz8ZXk8p/LRDzWPW6dG3v76hYpo9JCuumrNFvEsXnjLrIG0fw9ddBhmUxaFZu0CdF4onO7gbj55LTcbgfo43+7fFTjcD9HG/3b4qwNKzXQt9xuB+jjf7t8VONwP0cb/dvirA0oi33G4H6ON/u3xU43A/Rxv92+KsDSiLfcbgfo43+7fFTjcD9HG/3b4qwNKIvULbSLCY4zGPLTIRkElFtMo+osDl6swKrsGtsKuZWD8qUE+aAy252+ira6nM/wCHXHqrAUrQRDcbQsHbOJ0mfKcvUXHzzXpWJ2OD2xymjx5eo/7MQ3cwbI/xqHxuB+jjf7t8VZnC9IpIV1H1Zotxil88ZdQJ2j+HqqbyG2vfxZuTyn8jL+Ax6kfo2nd9lTQDuydVWucz6gsxF3HvHMZq543A/Rxv92+KnG4H6ON/u3xVjsUwuS2bKZWXqO9W/ZYbDUKsyCLCt2uDhMGYSlKVCslKUoiUpSiJUvDcSktm1oWZT0gbQe9TsNRKVIJFoUOaHCRuWl8o295+NrxMvbxDYT1um3/XvFRMS0ckiXXi1Z4d4ki87Z+kozI+0DrqlqXhuJSWza0LMp6RvB/aU7D9daUw7tDiL/dc9S5n0zwNo6jmMlEpWl8o295su14mU/l4hsJ63T/Xb3iomJaOSRLrxas8XRJF5+z9IDaPtHrqDDN7bQpbtAnReKJzu4G71yVLSlKzXQlKUoiUpSiJSlKIrfCtIZIF1G1ZYcsjFJ54y6hntX+HqqbyO2vPxduTzH8lJtRj1I/RmTu9grN0rQRDcbQsHbOJ0mfKcvUXHzzU3FMLltmymVl6m3q3cw2Hu31Cq4wvSGSFdRtWWLcYpfPGXUM9q9271VM5HbXm23bk8p/JSnNGJ6EfozJ/lU0A7snVVrXM+oLMRdxF45jNZulTcUwuS2bKZWXqberfssNh7t9QqzIIMit2uDhMGYSlKVCslKUoiUpSiL72N28EiyQM8cinNXQlGU5ZZhl2jZXomH8JiXiLBpDBHexDYtwgEU8QPSpXINtA/BKdOZbdXmlKziQmvvFuhXVs22xdnmGGw3g2tPiDMHSeC9LvuDaO+jafR2dLxBta1kIhniBJG0NqhtoORITPLZrV55iFlJbyNHOskcinJkcFGU+sHbX8sbt4HEkDSRyDarxs0bKfUykEV6JYcJiXkawaQwR3sQGS3CBYriPpzVhqhujcUz6S1Zf7Yf8AIaO6Hkuv/wCJtP8A+T+Jhnzc3/2HgvNKV6Zf8Gsd6jTaPTx3aAazWshEdxGD0EHINtByLBM8tmtWY0a0Jub67e0VGimSKSR1mDRaojXzQwYZgu5VQf0s9wq7doYQTO6+dhWET/G7Qx7W0Z0rAW/MD4ETB1s71mqV+5oijMrhlZSVZWBUqynIqwO0MDsyNfitlwpSlKIlKUoiVMwvFJLZs4WZesbw3ep2GodKkEi0KrmhwkRMLS8utrz8ZXk8p/LRDNWPW6dGZ/qKhYpo9JCuumrNDvEsfnjLrIG0fw9dU9TMLxSS2bOFmXrXerftKdhrSmHdocQsKlzPpmzA2jh3jmMlDpWk5dbXn4yvJ5j+WiHmE9bp0bf6ioeKaPSQrrpqzRbxLF54y62A2r/D11Bhm8WhS3aBOi8UTndwNx88lT0pSs10JSlKIlKUoiUpSiK4wvSGSFdR9WWHcYpPPGXUCdo/h6qm8itrz8Wbk8x/IybUY9SP0Zk7vsrNUrQRDcbQud2zidJhonL1Fx881ruaccznkk0bDfqnJyv1qdo9eQqLNoyiMVe4tlYb1bzSO8E51nYpCpBUkEbQQSCD6iNoq/t9I+MUJfos6bg/4Mi9zDLP7M+kmtA6GbxI8lg+HHZc6Y8BPykeSc3ovzm09v8AOnN6L85tPb/Ov7Lo+s4L4e4lG8wvkki+3IN9n11Q3EDRsVkDKw3hgVPsNQ6Tb281aGXPuiGeEgD5K95vRfnNp7f505vRfnNp7f51nqVWm3d81rVRN86DotDzei/ObT2/zpzei/ObT2/zrPUpTbupVRN86DotDzei/ObT2/zpzei/ObT2/wA6z1KU27qVUTfOg6LQ83ovzm09v86l4bYcmbWhu7ZT0gEEHvUnI/XWTpQRGi0DmVV0B7hIvs8B0W9vrW1uVznktFl7WFgmsetkY5H+PrFVw0NzTXWaEx79cAkZdeYJFZOpeGYlJbNrQsynpA2g/tKdh+ur1rHH5m6LIbNFY2UN+t3t+2K35vRfnNp7f505vRfnNp7f51+/KNvefja8TKfy8X4JPW6f67T6xUTEtHJIl14tWeLeJIvO2fpKNo+0eupIbe1sxxUNe+dF7y05gS4GUvXJSOb0X5zae3+dOb0X5zae3+dZ2lZ027vmuiqib50HRaLm9F+c2nt/nTm9F+c2nt/nWdpSm3d80qom+dB0Wi5vRfnNp7f505vRfnNp7f51naUpt3fNKqJvnQdFoub0X5zae3+dObsf5zae0e+s7SlNu75pVRN86DotzhY4ldRrmyliyyMUmTjLqBJzX+Hqr44jgNrMw5NLDG5P4GsJFJPQu0MP/mwVjKVeuBEi1Y/CODqQeQcgOeK1V3ogIcuNmgTPdrArn3Zmo/N2P85tPaPfUXCtIZIF1G1ZYdxik88ZdQz2rs6N3qqZyK2vPxZuTy9lKc0Y9SP0bf6VPyO7I1VCYzO24yxABHGyY5jNfnm7H+c2ntHvpzdj/ObT2j31VYphcls2UysvU29W7mGw92+oVZlzRYW+a3ax7hMRJjwC0XN2P85tPaPfTm7H+c2ntHvrO0pTbuq1VE3zoOi0XN2P85tPaPfTm7H+c2ntHvrO0pTbupVRN86DotFzdj/ObT2j305ux/nNp7R76ztKU27qVUTfOg6LUWGEiCRZILu3jkU5q6MUZTllmGVsxsJH110LoLpAsVjFJit1YvMwzErNBAeJYgxK+0ZnLzs8h+EM8zmTylSuPa9nZHaBKWd58F7X+E/ykb/HRHPmXgiUjY2dnzWC0i4eK9v4UdDrfF7o3WETWssjZcpihZJzrZZLMFjJIDAZN6wDvJNeb3eiiwuySz28bqcmR80ZT1FWIIrO2tw0Th4mdHBzV0JRlPWGXaD3VubLhBF0ghxyFLyIbFnXKK4j6fNdcg3RszXPpJrXZ2iEwMNssVx/5GM7ao7ozfkpWkC0T7zaO+85qj5ux/nNp7R76c3Y/wA5tPaPfV7d6Ax3iNLgUyXSgZtayFYriMbtoOQfbntyXPLZrVh760eBykyvG43o4KMO8Ntrem3dXDVRN86Dorvm7H+c2ntHvpzdj/ObT2j31naUpt3fNKqJvnQdFoubsf5zae0e+nN2P85tPaPfWdpSm3d80qom+dB0Wi5ux/nNp7R76c3Y/wA5tPaPfWdpSm3d80qom+dB0Wi5ux/nNp7R76mYXY8mbOG7tV6xmCrd6k5GsjSpERotDeZVXQHuEi+Y8B0W8v7K1uVzlktEm6ZIWVQx62Rjlv8AXn66rzob5mvx0PF5Z6+R1cuvMHLKsnUzC8UktmzhZl6xvVu9TsNWrWOPzN0WXw0VjZQ36/pkrbm7H+c2ntHvpzdj/ObT2j31+uXW17+MryeY/lotqMet06Nv9RUPE9HpIRrpqzRbxLF54y9YG0fw9dSQ28NmOKhr3zoveWnMCXA3HzyUrm7H+c2ntHvpzdj/ADm09o99Z2lZ027q6KqJvnQdFoubsf5zae0e+nN2P85tPaPfWdpSm3dSqib50HRaLm7H+c2ntHvpzdj/ADm09o99Z2lKbd1KqJvnQdEpSlZLoX7ikKEMhKsNxBII7iNoq+t9IxIoS/RZ03B9iyKPUwyz+z1ms9SrNeW3LOJCa+8dVopdHlnBfD3Eo3mJ8kkX25Bu/Z9dUNxC0bFZAysN4YFSPqNfmKQqQVJUjaCCQR3EbRV/BpHxqhL9FnToceZIvcwyz+z1mr/I7I8vZZf7Yf8AIaO6Hks9StFLo+s4L4e4lG8wvkki+3IN37PrqhuIWjYrIGVhvVgVI+o1VzC29aQ4zX3X4XFfOlKVRapSlKIlKUoiVLw3EpLZs4WZesDaD+0p2H66iUqQSLQoc0OEiLFpfKNvebLpeIl7eIeaT1un+u0+sVExLRySJdePVmi3iSLztn6SjaPtHrqlqXhmJSWza0LMp6RvB71Ow1pTDu0OIv8Adc9S5n0zwNo6jmMlEpWl8o295su14mU/l4vwSet0/r3iomJaOSRLrx6s0W8SReds/SUbR9o9dQYZvbaFLdoE6LxROd3A3euSpaUpWa6EpSlESlKURKUpRFcYXpDJCNR8potxil88ZdQJ2r3bvVUzkVte/izcnmP5GTahPUj9G3+grN0rQRDcbQsHbOJ0mfKcvUXHzzU3FMLktmymVl6m3q3cw2Hu31Cq4wvSGWFdRtWWHcYpPPGXUCdo2fV6qmchtrz8Wbk8vYynNGP6D9G3+gqaAd2TwKrXOZ9QWYi7j3jmM1m6VNxTC5LZsplZept6t3MNh7t9QqzIIsK3a4OEwZhKUpUKyUpSiJSlKIvraXDROrxM6OpzV0JRlO7NWXIg91bmy4QFuUEOOQx3kYGSzqBFcRg+i4yD9GzNc8tpNYGlEW/udAEvEMuBTJdKBm1tIRDcR9G0NkrjftyUbNmtWGvbV4XKTK8bjYyOCjDvVsiK/lrcNE4eJnRwc1dCUZT1hl2g91bmz4QVukEOOwpeRjYs65Qzx+tXXVD9GzNc+kmiLA0rf3WgKXiNLgUyXSja1rIRDcRjdkVbIOM89vm7tmtWGvbV4HaOZXjkU5MjgoynLParbRsoi+NKUoiUpSiJSlKIlTcLxSS2OcLMvWu9W71Ow1CpUgkWhVc0OEiJhaTlttebLleTyn8tEM0YnpdOjb/UVDxTR6SFdddWWLeJY/PGXWQNq/w9dU9TcLxSS2bOFmXrG9W71Ow9++tKYd2hxCwqXM+mbMDaOHeOYyUKlaTltte/jK8nmP5aPajHrdOjb/WoeKaPSQrrrqyxbxLF54y6yBtH8PXUGGbxaFLdoE6LxROfobj55KnpSlZroV1zVuezPij+KnNW57M+KP4qrOWP6Unib305Y/pSeJvfWs4eB19lzyj4t0P3Kz5q3PZnxR/FTmrc9mfFH8VVnLH9KTxN76csf0pPE3vpOHgdfZJR8W6H7lZ81bnsz4o/ipzVuezPij+Kqzlj+lJ4m99OWP6Unib30nDwOvsko+LdD9ytYtGbtCCqOrDcQyAjuIbMVf28FzIoS+gE6bg+cayL3Nrbfs76xfLH9KTxN76csf0pPE3vqzYjW3T/AHgsokCI+8t0M/8AktPiuhbga1rmw7OTJXHqz/Ab7Prqr5rXPZnxR/FVat7INoeUHr1m99XlnpQWXUvF41Oh1PFyL6wy5a32esmpnCcbiPLyUEbTDFhDuFvmAVF5rXPZnxR/FTmtc9mfFH8VT5cKM4L4fM8o3mFmKSKPrOTd+z66oZ5pY2KuZlYb1YupHeDUOY1t4OvsphxYj7nNnhIg/wDJT+a1z2Z8UfxU5rXPZnxR/FVZyx/Sk8Te+nLH9OTxN76rOHgdfZayj4t0P3Kz5rXPZnxR/FTmtc9mfFH8VVnLH9OTxN76csf05PE3vpOHgdfZJR8W6H7lZ81rnsz4o/ipzWuezPij+Kqzlj+nJ4m99OWP6cnib30nDwOvsko+LdD9ys+a1z2Z8UfxVLwzCL62bWhV16xrRkH9pS2Rqh5Y/pyeJvfTlj+nJ4m99A5gtE9fZVcyM4SJbLwP3LbnBzefjcDQy9tCUyJ63TWJ/ifWKo8Q0OnjbKMCVehlKr7QxBB7s++qTlj+nJ4m99feyxeaFtaN5AeoksD6iGzBq5iQ3Xg+vksmwI7Oy4SwM5eZI8slM5rXPZnxR/FTmtc9mfFH8VWC41Fd7LnjIJO2iLapPW6f17xUXEsJuIl143eaHeJImZtn6QBzH2j11JhslNsyPH2UNjxZ0XlrTmDLgaUvXJfHmtc9mfFH8VOa1z2Z8UfxVWcsf05PE3vpyx/Tk8Te+s5w8Dr7LolHxbofuVnzWuezPij+KnNa57M+KP4qrOWP6cnib305Y/pyeJvfScPA6+ySj4t0P3Kz5rXPZnxR/FTmtc9mfFH8VVnLH9OTxN76csf05PE3vpOHgdfZJR8W6H7lZ81rnsz4o/ipzWuezPij+Kqzlj+nJ4m99OWP6cnib30nDwOvsko+LdD9y1eF299Cuo8fHRbjFKY3GXUCWzGz6vVX2u9EROpeJXt5OmJyJEJ/RZCSBn1+wVjuWP6cnib305Y/pyeJvfVxFZKRBPj/ANLA7NFnSa4A5A87Zcpq0fRW5BI4sn1ho9vrHnbq/nNa57M+KP4qYVpLLB5rHjY+mOTzth6mO0fw9VWOrFebbaWS3lP5GVmKMepHz2bf6CgZDddOfj7I6LHYfmlLEAnW2Y0lmq7mtc9mfFH8VOa1z2Z8UfxV8MTguLZspuOXqOsxB7mByNQ+WSenJ4m99UNAWEH94LVpiuEw5pHgfuVnzWuezPij+KnNa57M+KP4qrOWSenJ4m99OWSenJ4m99Jw8Dr7K0o+LdD9ys+a1z2Z8UfxU5rXPZt4o/iqs5ZJ6cnib305ZJ6cnib30nDwOvsko+LdD9ys+a1z2beKP4qc1rns28UfxVWcsk9OTxN76csk9OTxN76Th4HX2SUfFuh+5W9ro9eROHiWVHBzV0dUZTuzDKwIPdW5ssYuLlBDjlqt5GBkJ1MUdxGDntVwwDdGzzc+kmvL+WSenJ4m99OWSenJ4m99Jw8Dr7JKPi3Q/ctxpRoAiLxmGy8amWfETgxTJ6tbIRP09K+rOsyNF7nsz4o/iqvjv5FIKvMCDmCGYEEbiCDsNbbCuEMTIIcYiW6j6J48reeP1hk1Q/RszXPLaTScPA6+ySj4t0P3LNc1rns28UfxU5rXPZt4o/irWXehvLEaXArk3SjabV2MNxEN20MQHAOe3Jc+jWrDXrTQuyTG4jkXYyProynftVsiKTh4HX2SUfFuh+5Tea1z2beKP4qc1rns28UfxVWcsk9OTxN76csk9OTxN76Th4HX2SUfFuh+5WfNa57NvFH8VOa1z2beKP4qreWyenL4m99OWyenL4m99Jw8Dr7JKPi3Q/crLmtc9m3ij+KnNa57NvFH8VVvLZPTl8Te+nLZPTl8Te+k4eB19klHxbofuVlzWuezbxR/FUzC8KvrZs4VdetdaMq3epbI/wAaoeWyenL4m99OWyenL4m99A5gtE9fZVcyK4SJaR4H7ltWwXln4zC1vL2sRQox/TTPPf1e2qO+0PuI2yQCRehlKr7QxBB9vfVNy2T05fE3vqRYYzNA2tG7+sElwfUQ2Yq5iQ3Xg/vBZNgx2dlw8DOXmT6L8YrhUtq2UysvU29W7mGw92+oVW+FaQSQLqHVlh3GKTz1y6hntXZ9XqqbyS2vP9w3Jpj+SkOaMT0I/RmT7lqlAO7J4FbVr2fUFmIu4i8cxms3SpuKYVLbHKZWXqberdzDYe7fUKsyCLCt2uDhMGYSlKVCslKUoiUpSiL9xSFCCpKsNxBII7iNoq+t9IxKoS/RZ03Bx5si9zDLP7PWTWepVmvLblnEgtfeOq0Uuj6zgvh7iUbzE+SSL9RyDD17PrqhnhaNirhlYb1YFSPqNfmKQoQVJVhtBBII7iNoq/g0jEqhL9FnXcH2LIvcwyz6Or1k1f5HZHl7LL/bD/kNHdDyWepWil0eWcF8PcSjeYmySRR9eQbv2fXVDPC0bFXDKw3qwKkfUaq5hbetIcZr7r8LivnSlKotUpSlESlKURKl4ZiUls2tCzL1jeD+0p2GolKkEi0KHNDhIixaXyjbXn42vES9tEPNJ63T+veKh4no5JEuvHqzRbxJF5+z9IDaPtHrqmqZhmJSWza0LMvWN4Pep2GtKYd2hxF/uuepcz6Z4G0dRzGSh0rS+ULe8/Gl4iU/l4vwSet0/r3ioeJ6OyRLrx6s0O8SR+cMvWBtH2j11Bhm9toUt2gTk8UTndwNx88lTUpSs10JSlKIlKUoiUpSiK4wzSKSFdR9WaLcYpfPGXUCdo/h6qm8htr38Wbk8p/Iy/gMepH6NvR9grNUrQRDcbQud2zidJhonK7iLj55qZimGSWzZTKy9R3q37LDYah1cYXpFJCuo+rNDuMUnnDLqBO0fw9VTeQ215+LNyeU/kZNqsepH6NvR9gqaAd2TwP7aornM+oLMRaOPeOYzWapUzFMMktmymVl6jvDdzDYah1mQRYV0NcHCYMwlKUqFKUpSiJSlKIvra3DROHiZ0cHNXQlGU7swy5EHurc2fCAt0iw47Cl5GNgnUCK4jG/NXXIPuGzNc8tpasDSiLf3egKXimXAZkukAza2kIiuI//ACkAON+3zc8tmtWGvLV4XKTK8bjYyOCjKcs9qtkRsr+Wtw0Tq8TOjqc1dCUZT1qy7Qe6tzZcIK3SLDjkKXkY2LOuUVxGDt811yDbhszXPLaTRFgaVv7vQBLtDLgUy3aja1tJqw3EY3bVbVDjftyXPLZrVhr21eF2jmV45FOTI4KMp35FTtFEXxpSlESlKURKUpREpWi55XHXH4RTnlcdcfhFa0WYnT3XPTjbo/t+KiYXpBLANRspYjsMUvnjLqGe1dnRu9VTeR215tt25NN2UpzRiehH6Npy/wD21+eeVx1x+EU55XHXH4RWgcyUiZjw91zuhxZ0mtDTk7zFGR881VYphclscplZept6t+yw2Hu31CrSwaaThgW4pl6V1dXP6xtBq+g0i5UALeRYJuzmVWVj1LIOs9e31VAhw3dl37qpdHjwwKbB4g2eRPovPKVr8Ux69tWymCL1Nqgq3cw2Hu31C55XHXH4RVSxgsJOnutWxYrhMNBH/l+KztK0XPK464/CKc8rjrj8IqKLMTp7q1ONuj+34rO0rRc8rjrj8IpzyuOuPwilFmJ090pxt0f2/FZ2laLnlcdcfhFOeVx1x+EUosxOnulONuj+34qgikKEMhKsNxBII7iNoq+g0jEqhL9FnTcHGSyL3MMs+jq9ZNf3nlcdcfhFOeVx1x+EVZpa25x091lEbEfewf2t/wCKS6PrOC9g6yjeYmySRfqOQbv2fXVDPC0bFXDKw3qwKkd4O2r7nlcdcfhFTMM03dT/ALQquvWnmMv1fgt3bO+pIhO75cLPNVa7aWAzaHDxt8gDosjSvQ5MRmnUvh8kMoG0xOqpIv1HIN37Prqhn0suo2KvqKw2FSgBH1GodCa28nT3Voe0xH2Bon4yP/FZqlaLnlcdcfhFOeVx1x+EVWizE6e61pxt0f2/FZ2laLnlcdcfhFOeVx1x+EUosxOnulONuj+34rO0rRc8rjrj8IpzyuOuPwilFmJ090pxt0f2/FZ2pmGYlJbNrQsy9Y3g/tKdhq255XHXH4RTnlcdcfhFSAwWhx091VxiuEiwS8fxX78oW95sul4iU/lovwSet0/1294qHiWjskS68erPF0SReeMh6QG0faPXUnnlcdcfhFOeVx1x+EVcuhu7Rt8JeqxayOw/IBLAmY4fLMayyWdpW6w3TISrqz5wv0SqBIuf6SnMgd2feK/uJ4hexLrxmGaHeJI1DDLrIG0faPXUVLZTBn4D3U/FRA6i5gBzPl8svVYSlaLnlcdcfhFOeVx1x+EVSizE6e63pxt0f2/FZ2laLnlcdcfhFOeVx1x+EUosxOnulONuj+34rO0rRc8rjrj8IpzyuOuPwilFmJ090pxt0f2/FZ2laLnlcdcfhFOeVx1x+EUosxOnulONuj+34qNhmkUkK6j6s0W4xy+eMuoE7R/D1VLNjbXm22bk8vYynzGJ6Ef1no+wV+eeVx1x+EU55XHXH4RWgcyUiZjw91zuhRZ0mNDTkbOIoyPnmqnE8MktmymVl6jvVv2WGw1DrTW+ms6sC/FOvSuWrn3EbjV9DpBysDk0iQy9lMqlWPUrj+Z9VQIcN3Zd+6qXbRGhgU2DxBs8ifRed0rXYpj97bNqzBF6jqgg9zDYah88rjrj8IqpYwWEnT3WrYsVwmGgj/y/FZ2laLnlcdcfhFOeVx1x+EVFFmJ091anG3R/b8VnaVoueVx1x+EU55XHXH4RSizE6e6U426P7fis7StFzyuOuPwinPK464/CKUWYnT3SnG3R/b8VRWtw0Tq8TOjqc1dCUZT1qy5EHurc2fCAt0iw47Cl5GBks6gRXEY3+a65Btw2ZrnltLVR88rjrj8IpzyuOuPwilFmJ090pxt0f2/FXt1oCl4jS4FMl0gGZtpCIriMbsipyD7c9uS59GtvrDXto8DlJlkjcb0dWjYd6sARV5zyuOuPwirnRnT1UbLEYuOQn8OJmikQZdC58XIN2w6vTtO6lFmJ090pxt0f2/FYSlewXNlLeIZcCngu0Aza3kVIbiMbtqnVVxnrbcl3DLWrD3mlF3C7RzBUkU5MjpqMp35EHaKUWYnT3SnG3R/b8Vl6VoueVx1x+EU55XHXH4RSizE6e6U426P7fis7SlKyXQlKUoiUpSiK3wrSCWBdQ6ssW4xSeeuWwZDPaoy6N3qqbyO2vPxduTzH8lIc0YnoR+jaf5Vm6VoIhlI2hYO2cTpN+U5eouPnmpuKYXJbNlMrL1NvVu5hsPdvqFVvhekEkI1Gyli3GKXz1y2bBntXu3eqpvIra9/F25PMfyMhzRj1I/Rt/pU0A7sngVWtcz6gsxF3EXjmM1m6VNxTC5LZsplK9Tb1buYbD3b6hVmQQZFbtcHCYMwlKUqFZKUpREpSlEX6ikKEFSVYbQQSCO4jaKv4NIxKoS/RZ03Bx5si9zDLP7PWTWepVmvLblnEhNfeOq0Uujyzgvh7iUbzE2SSL9RyDd+z66obiFo2KuGVhvDAqR9Rr+RSFCGQsrDcQSpHcRtFX0GkQlUJfos67hIMlkXuYZZ/Z6yav8jsjy9ll/th/wAho7oeSz1K0Uujyzgvh7iUbzE2SSL9RyDdO3Z9dUM8LRsVcMrDerAqR9RqrmFt60hxmvuvwuK+dKUqi1SlKURKUpREqZhmJSWza0LMvWN4b9pTsNQ6VIJFoUOaHCRFi0vlC3vPxpeIlP5aL8EnrdP694qHiejskS68erNFvEkXn7PWBtH2j11TVMwzEpLZs4WZesbwf2lOw1pTDu0OIv8Adc9S5n0zwNo4d45jJQ6VpfKFtebLpeIl7aIeaT1un+u3vFQ8T0ckiXXj1Zot4ki87Z+kBtH2j11Bhm9toUt2gTovFE53cDcfPJU1KUrNdCUpSiJSlKIlKUoiucL0ikhXUfVmh3GKXzxl1KTtH2j1VM5BbXn4q3ES9jKfNY9SP39/cKzVK0EQ3OtC53bOJ0mGicruIuPnmpmJ4ZJbNlMrL1HeG/ZYbDUOrnDNIpIl1JNWaLcYpfPGX6JO0faPVUzkFtebbVuIlP5GX8EnqR/69wqaAd2TwP7aornM+oLMRaOPeOYzWapUzE8MktmymVl6jvB7mGw1DrMgiwroa4OExaEpSlQpSlKURKUpRF9bW4aJ1eJnRwc1dCUZT1qy7Qe6tzZ8IC3SLDjsKXkY2LOuUVxH0+a65Bxu2Zrn0lqwNKIt9d6Apdo0uBTJdoBm1tIViuIxu2qdUPtz25Lnls1qw97aPA5SZXjcb0cFGHeG2iv5aXLQurxM6OpzV0JRlO7NWXIg91bmy0/W6RYcchjvIwNVbhQIriIHbmrrkH6Nma55bS1EWBpVxeY7xza0kNmW6Tqyrn36sgzPrNfHymvY2n3/AM2rloxWQe+VreYVbSrLymvY2n3/AM2nlNextPv/AJtKIxU03bp5dVW0qy8pr2Np9/8ANp5TXsbT7/5tKIxSm7dPLqq2lWXlNextPv8A5tPKa9jaff8AzaURilN26eXVVtKsvKa9jaff/Np5TXsbT7/5tKIxSm7dPLqvrhekMsK6jassO4xSDXGXUCdq7Pq9VTeRW17+LtyeXspTmjH9B+jb/Sq3ymvY2n3/AM2nlNextPv/AJtaB3cTMLndCM6TAWnKXMTkfPNfLFMLltmymVl6m3q3cw2Hu31CrQQ6VSKnFhLYx7tRhLIMurJ5DkPVUE4ovY2nsm+ZVHNZ3FaQ3xZfM3T/ALs1KraVZeU17G0+/wDm08pr2Np9/wDNqKIxWlN26eXVVtKsvKa9jaff/Np5TXsbT7/5tKIxSm7dPLqq2lWXlNextPv/AJtPKa9jaff/ADaURilN26eXVVtKsvKa9jaff/Np5TXsbT7/AObSiMUpu3Ty6qBFIUIKkqw3EEgjuI2ir6DSMSqEv0WdOhx5ki9zDLPu2esmoHlNextPv/m08pr2Np9/82rNNG4rKIyne3r5qwl0eWcF8PdZRvMTZJIv1HIN07dnqzqhnhaNirhlYb1YFSPqNWEWLhCCsVsrDcV5QpHcRLmKlX2k7zgCaO0cDcWWQkdza+t9tS6gcjyVWGM0yImM5A9DyVDSrLymvY2n3/zaeU17G0+/+bVKIxW9N26eXVVtKsvKa9jaff8AzaeU17G0+/8Am0ojFKbt08uqraVZeU17G0+/+bTymvY2n3/zaURilN26eXVVtKsvKa9jaff/ADaeU17G0+/+bSiMUpu3Ty6qtqZhmJSWza0LMvWN4Pep2Gvt5TXsbT7/AObTymvY2n3/AM2gAFoKq4lwkWzHDqrPyhb3n40vESn8vF+CT1un9e8VDxPR2SJdePVmi3iSLzhl+kBtH2j118PKa9jaff8Azak2GkbQHOGO3Try4/I96mTI/WK1pNd2uX7asKERn0wZYG0cLZjmMlSUq3usb41izw2ZY7yFlXPv1ZACfXXy8pr2Np9/82sqIxXQHu728wq2lWXlNextPv8A5tPKa9jaff8AzaURippu3Ty6qtpVl5TXsbT7/wCbTymvY2n3/wA2lEYpTdunl1VbSrLymvY2n3/zaeU17G0+/wDm0ojFKbt08uqraVZeU17G0+/+bTymvY2n3/zaURilN26eXVffDNIpIV1H1Zodxik84ZdQJ2j7R6qmcgtrzbatxEvYynzWPUj/ANe4VWeU17G0+/8Am08pr2Np9/8ANrQP7nGYXO6EZ0mAtOUpcROR8818cTwyS2bKZWXqO8HuYbDUOtAmlcgTi9S2KeiwlkGXV58hyHqqD5TXsbT7/wCZVHNZ3FaMfFl8zdP2zUqtpVl5TXsbT7/5tPKa9jaff/NqKIxWlN26eXVVtKsvKa9jaff/ADaeU17G0+/+bSiMUpu3Ty6qtpVl5TXsbT7/AObTymvY2n3/AM2lEYpTdunl1VbSrLymvY2n3/zaeU17G0+/+bSiMUpu3Ty6qtpSlUWiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIlKUoiUpSiJSlKIv/9k=\n", "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from IPython.display import YouTubeVideo\n", "\n", "YouTubeVideo(\"kYB8IZa5AuE\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## British Literature SVD & NMF in Excel" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Data was downloaded from [here](https://de.dariah.eu/tatom/datasets.html)\n", "\n", "The code below was used to create the matrices which are displayed in the SVD and NMF of British Literature excel workbook. The data is intended to be viewed in Excel, I've just included the code here for thoroughness." ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "### Initializing, create document-term matrix" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "import numpy as np\n", "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n", "from sklearn import decomposition\n", "from glob import glob\n", "import os" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "np.set_printoptions(suppress=True)" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "filenames = []\n", "for folder in [\"british-fiction-corpus\"]: #, \"french-plays\", \"hugo-les-misérables\"]:\n", " filenames.extend(glob(\"data/literature/\" + folder + \"/*.txt\"))" ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "27" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(filenames)" ] }, { "cell_type": "code", "execution_count": 134, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "((27, 55035), 55035)" ] }, "execution_count": 134, "metadata": {}, "output_type": "execute_result" } ], "source": [ "vectorizer = TfidfVectorizer(input='filename', stop_words='english')\n", "dtm = vectorizer.fit_transform(filenames).toarray()\n", "vocab = np.array(vectorizer.get_feature_names())\n", "dtm.shape, len(vocab)" ] }, { "cell_type": "code", "execution_count": 135, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "['Sterne_Tristram.txt',\n", " 'Austen_Pride.txt',\n", " 'Thackeray_Pendennis.txt',\n", " 'ABronte_Agnes.txt',\n", " 'Austen_Sense.txt',\n", " 'Thackeray_Vanity.txt',\n", " 'Trollope_Barchester.txt',\n", " 'Fielding_Tom.txt',\n", " 'Dickens_Bleak.txt',\n", " 'Eliot_Mill.txt',\n", " 'EBronte_Wuthering.txt',\n", " 'Eliot_Middlemarch.txt',\n", " 'Fielding_Joseph.txt',\n", " 'ABronte_Tenant.txt',\n", " 'Austen_Emma.txt',\n", " 'Trollope_Prime.txt',\n", " 'CBronte_Villette.txt',\n", " 'CBronte_Jane.txt',\n", " 'Richardson_Clarissa.txt',\n", " 'CBronte_Professor.txt',\n", " 'Dickens_Hard.txt',\n", " 'Eliot_Adam.txt',\n", " 'Dickens_David.txt',\n", " 'Trollope_Phineas.txt',\n", " 'Richardson_Pamela.txt',\n", " 'Sterne_Sentimental.txt',\n", " 'Thackeray_Barry.txt']" ] }, "execution_count": 135, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[f.split(\"/\")[3] for f in filenames]" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true }, "source": [ "### NMF" ] }, { "cell_type": "code", "execution_count": 136, "metadata": { "hidden": true }, "outputs": [], "source": [ "clf = decomposition.NMF(n_components=10, random_state=1)\n", "\n", "W1 = clf.fit_transform(dtm)\n", "H1 = clf.components_" ] }, { "cell_type": "code", "execution_count": 137, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "num_top_words=8\n", "\n", "def show_topics(a):\n", " top_words = lambda t: [vocab[i] for i in np.argsort(t)[:-num_top_words-1:-1]]\n", " topic_words = ([top_words(t) for t in a])\n", " return [' '.join(t) for t in topic_words]" ] }, { "cell_type": "code", "execution_count": 138, "metadata": { "hidden": true }, "outputs": [], "source": [ "def get_all_topic_words(H):\n", " top_indices = lambda t: {i for i in np.argsort(t)[:-num_top_words-1:-1]}\n", " topic_indices = [top_indices(t) for t in H]\n", " return sorted(set.union(*topic_indices))" ] }, { "cell_type": "code", "execution_count": 139, "metadata": { "hidden": true }, "outputs": [], "source": [ "ind = get_all_topic_words(H1)" ] }, { "cell_type": "code", "execution_count": 140, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/plain": [ "array(['adams', 'allworthy', 'bounderby', 'brandon', 'catherine', 'cathy',\n", " 'corporal', 'crawley', 'darcy', 'dashwood', 'did', 'earnshaw',\n", " 'edgar', 'elinor', 'emma', 'father', 'ferrars', 'finn', 'glegg',\n", " 'good', 'gradgrind', 'hareton', 'heathcliff', 'jennings', 'jones',\n", " 'joseph', 'know', 'lady', 'laura', 'like', 'linton', 'little', 'll',\n", " 'lopez', 'louisa', 'lyndon', 'maggie', 'man', 'marianne', 'miss',\n", " 'mr', 'mrs', 'old', 'osborne', 'pendennis', 'philip', 'phineas',\n", " 'quoth', 'said', 'sissy', 'sophia', 'sparsit', 'stephen', 'thought',\n", " 'time', 'tis', 'toby', 'tom', 'trim', 'tulliver', 'uncle', 'wakem',\n", " 'wharton', 'willoughby'], \n", " dtype='britlit_W.csv
" ], "text/plain": [ "/data/jhoward/rachel/num-lin-alg/nbs/britlit_W.csv" ] }, "execution_count": 119, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.savetxt(\"britlit_W.csv\", W1, delimiter=\",\", fmt='%.14f')\n", "FileLink('britlit_W.csv')" ] }, { "cell_type": "code", "execution_count": 120, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/html": [ "britlit_H.csv
" ], "text/plain": [ "/data/jhoward/rachel/num-lin-alg/nbs/britlit_H.csv" ] }, "execution_count": 120, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.savetxt(\"britlit_H.csv\", H1[:,ind], delimiter=\",\", fmt='%.14f')\n", "FileLink('britlit_H.csv')" ] }, { "cell_type": "code", "execution_count": 131, "metadata": { "hidden": true }, "outputs": [ { "data": { "text/html": [ "britlit_raw.csv
" ], "text/plain": [ "/data/jhoward/rachel/num-lin-alg/nbs/britlit_raw.csv" ] }, "execution_count": 131, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.savetxt(\"britlit_raw.csv\", dtm[:,ind], delimiter=\",\", fmt='%.14f')\n", "FileLink('britlit_raw.csv')" ] }, { "cell_type": "code", "execution_count": 121, "metadata": { "collapsed": true, "hidden": true }, "outputs": [ { "data": { "text/plain": [ "['adams',\n", " 'allworthy',\n", " 'bounderby',\n", " 'brandon',\n", " 'catherine',\n", " 'cathy',\n", " 'corporal',\n", " 'crawley',\n", " 'darcy',\n", " 'dashwood',\n", " 'did',\n", " 'earnshaw',\n", " 'edgar',\n", " 'elinor',\n", " 'emma',\n", " 'father',\n", " 'ferrars',\n", " 'finn',\n", " 'glegg',\n", " 'good',\n", " 'gradgrind',\n", " 'hareton',\n", " 'heathcliff',\n", " 'jennings',\n", " 'jones',\n", " 'joseph',\n", " 'know',\n", " 'lady',\n", " 'laura',\n", " 'like',\n", " 'linton',\n", " 'little',\n", " 'll',\n", " 'lopez',\n", " 'louisa',\n", " 'lyndon',\n", " 'maggie',\n", " 'man',\n", " 'marianne',\n", " 'miss',\n", " 'mr',\n", " 'mrs',\n", " 'old',\n", " 'osborne',\n", " 'pendennis',\n", " 'philip',\n", " 'phineas',\n", " 'quoth',\n", " 'said',\n", " 'sissy',\n", " 'sophia',\n", " 'sparsit',\n", " 'stephen',\n", " 'thought',\n", " 'time',\n", " 'tis',\n", " 'toby',\n", " 'tom',\n", " 'trim',\n", " 'tulliver',\n", " 'uncle',\n", " 'wakem',\n", " 'wharton',\n", " 'willoughby']" ] }, "execution_count": 121, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[str(word) for word in vocab[ind]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### SVD" ] }, { "cell_type": "code", "execution_count": 143, "metadata": { "collapsed": true }, "outputs": [], "source": [ "U, s, V = decomposition.randomized_svd(dtm, 10)" ] }, { "cell_type": "code", "execution_count": 144, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ind = get_all_topic_words(V)" ] }, { "cell_type": "code", "execution_count": 145, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "52" ] }, "execution_count": 145, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(ind)" ] }, { "cell_type": "code", "execution_count": 146, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['adams', 'allworthy', 'bounderby', 'bretton', 'catherine',\n", " 'crimsworth', 'darcy', 'dashwood', 'did', 'elinor', 'elton', 'emma',\n", " 'finn', 'fleur', 'glegg', 'good', 'gradgrind', 'hareton', 'hath',\n", " 'heathcliff', 'hunsden', 'jennings', 'jones', 'joseph', 'knightley',\n", " 'know', 'lady', 'linton', 'little', 'lopez', 'louisa', 'lydgate',\n", " 'madame', 'maggie', 'man', 'marianne', 'miss', 'monsieur', 'mr',\n", " 'mrs', 'pelet', 'philip', 'phineas', 'said', 'sissy', 'sophia',\n", " 'sparsit', 'toby', 'tom', 'tulliver', 'uncle', 'weston'], \n", " dtype='britlit_U.csv
" ], "text/plain": [ "/data/jhoward/rachel/num-lin-alg/nbs/britlit_U.csv" ] }, "execution_count": 148, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.savetxt(\"britlit_U.csv\", U, delimiter=\",\", fmt='%.14f')\n", "FileLink('britlit_U.csv')" ] }, { "cell_type": "code", "execution_count": 149, "metadata": {}, "outputs": [ { "data": { "text/html": [ "britlit_V.csv
" ], "text/plain": [ "/data/jhoward/rachel/num-lin-alg/nbs/britlit_V.csv" ] }, "execution_count": 149, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.savetxt(\"britlit_V.csv\", V[:,ind], delimiter=\",\", fmt='%.14f')\n", "FileLink('britlit_V.csv')" ] }, { "cell_type": "code", "execution_count": 150, "metadata": {}, "outputs": [ { "data": { "text/html": [ "britlit_raw_svd.csv
" ], "text/plain": [ "/data/jhoward/rachel/num-lin-alg/nbs/britlit_raw_svd.csv" ] }, "execution_count": 150, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.savetxt(\"britlit_raw_svd.csv\", dtm[:,ind], delimiter=\",\", fmt='%.14f')\n", "FileLink('britlit_raw_svd.csv')" ] }, { "cell_type": "code", "execution_count": 151, "metadata": {}, "outputs": [ { "data": { "text/html": [ "britlit_S.csv
" ], "text/plain": [ "/data/jhoward/rachel/num-lin-alg/nbs/britlit_S.csv" ] }, "execution_count": 151, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.savetxt(\"britlit_S.csv\", np.diag(s), delimiter=\",\", fmt='%.14f')\n", "FileLink('britlit_S.csv')" ] }, { "cell_type": "code", "execution_count": 152, "metadata": { "collapsed": true }, "outputs": [ { "data": { "text/plain": [ "['adams',\n", " 'allworthy',\n", " 'bounderby',\n", " 'bretton',\n", " 'catherine',\n", " 'crimsworth',\n", " 'darcy',\n", " 'dashwood',\n", " 'did',\n", " 'elinor',\n", " 'elton',\n", " 'emma',\n", " 'finn',\n", " 'fleur',\n", " 'glegg',\n", " 'good',\n", " 'gradgrind',\n", " 'hareton',\n", " 'hath',\n", " 'heathcliff',\n", " 'hunsden',\n", " 'jennings',\n", " 'jones',\n", " 'joseph',\n", " 'knightley',\n", " 'know',\n", " 'lady',\n", " 'linton',\n", " 'little',\n", " 'lopez',\n", " 'louisa',\n", " 'lydgate',\n", " 'madame',\n", " 'maggie',\n", " 'man',\n", " 'marianne',\n", " 'miss',\n", " 'monsieur',\n", " 'mr',\n", " 'mrs',\n", " 'pelet',\n", " 'philip',\n", " 'phineas',\n", " 'said',\n", " 'sissy',\n", " 'sophia',\n", " 'sparsit',\n", " 'toby',\n", " 'tom',\n", " 'tulliver',\n", " 'uncle',\n", " 'weston']" ] }, "execution_count": 152, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[str(word) for word in vocab[ind]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Randomized SVD offers a speed up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One way to address this is to use randomized SVD. In the below chart, the error is the difference between A - U * S * V, that is, what you've failed to capture in your decomposition:\n", "\n", "\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For more on randomized SVD, check out my [PyBay 2017 talk](https://www.youtube.com/watch?v=7i6kBz1kZ-A&list=PLtmWHNX-gukLQlMvtRJ19s7-8MrnRV6h6&index=7).\n", "\n", "For significantly more on randomized SVD, check out the [Computational Linear Algebra course](https://github.com/fastai/numerical-linear-algebra)." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## Full vs Reduced SVD" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Remember how we were calling `np.linalg.svd(vectors, full_matrices=False)`? We set `full_matrices=False` to calculate the reduced SVD. For the full SVD, both U and V are **square** matrices, where the extra columns in U form an orthonormal basis (but zero out when multiplied by extra rows of zeros in S)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Diagrams from Trefethen:\n", "\n", "\"\"\n", "\n", "\"\"" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 2 }