{"cells":[{"cell_type":"markdown","metadata":{"id":"oapouz4hhs25"},"source":["# **Maigret**\n","<p align=\"center\">\n"," <p align=\"center\">\n"," <a href=\"https://pypi.org/project/maigret/\">\n"," <img alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/maigret?style=flat-square\">\n"," </a>\n"," <a href=\"https://pypi.org/project/maigret/\">\n"," <img alt=\"PyPI - Downloads\" src=\"https://img.shields.io/pypi/dw/maigret?style=flat-square\">\n"," </a>\n"," <a href=\"https://pypi.org/project/maigret/\">\n"," <img alt=\"Views\" src=\"https://komarev.com/ghpvc/?username=maigret&color=brightgreen&label=views&style=flat-square\">\n"," </a>\n"," </p>\n","</p>\n","\n","\n","# **Summary** 📋\n","Maigret collects a dossier (basically, a report) for a specified username, checking for accounts on a huge number of sites and gathering all the available information from web pages.\n","\n","\n","## **Features**\n","-----\n","- Extracts information from account profiles based on username\n","- By default, searches 500 sites for username, but search on 3000 sites possible\n","- Recursively searches based on related usernames found on account profiles\n","- Search can be narrowed to websites with specific tags relating to genre, language, etc.\n","- Generates HTML, PDF, xMind 8 Mindmaps, .txt, .csv, or JSON reports of results\n","- Retries requests in case of an error\n","- Also checks mirrors, archives, and aggregators to search for profiles that may have been taken down\n","- Detects captcha or censorship and outputs message to terminal accordingly\n","\n","## **Limitations**\n","-----\n","- Rate limits - some websites will only allow for a certain number of queries from an individual IP address\n","- Overhead - depending on search parameters, queries can take a long time\n"]},{"cell_type":"markdown","metadata":{"id":"iCxn0_uUFVWR"},"source":["## **Result**\n","---\n","The output dossier can have a variety of looks, here are a few examples 👇\n","\n"]},{"cell_type":"markdown","metadata":{"id":"60-QFBmvoJxC"},"source":["### Terminal\n","---\n","You'll see information for each individual website for which there is a match and a short summary at the end.\n","\n","Individual Website Report 👇\n","\n","\n","\n","Short Summary👇\n","\n",""]},{"cell_type":"markdown","metadata":{"id":"i6e_OQBaoN6O"},"source":["### File-based Report\n","---\n","You'll see the same information as in the terminal, but displayed in document form. Below is an example of a .pdf generated by maigret.\n","\n",""]},{"cell_type":"markdown","metadata":{"id":"Kd3W6ue-1RC7"},"source":["### **Note** 👇\n","Many of the code examples below included either '!'or '&&shell'. They are included purely for use in Jupyter Notebooks and are not required if you are running these commands on your computer at home."]},{"cell_type":"markdown","metadata":{"id":"DzGAgpz4FX0p"},"source":["# **Installation** 💿\n","You can install maigret using 'pip', a package installer for Python\n","\n","Run the following code snippet to install maigret:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"f9Ey4eIF5P_A"},"outputs":[],"source":["!pip install maigret"]},{"cell_type":"markdown","metadata":{"id":"SY61M7uI7ZEz"},"source":["#**Usage** 💻\n","## **Search for a username** 😎\n","---\n","You will enter your command in the following format:\n","\n","```maigret <username>```\n","\n","Run the following command to see how to query a username:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"8DwerqPlr-26"},"outputs":[],"source":["!maigret trump123"]},{"cell_type":"markdown","metadata":{"id":"YGsy8TeGlHoC"},"source":["## **Search for multiple usernames** 😎 😎 😎\n","-------\n","\n","You will enter your command in the following format:\n","\n","```maigret <username_1> <username_2> ... <username_3>```\n","\n","Run the following command to see how to query multiple usernames:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"a-YpxpdEnEz7"},"outputs":[],"source":["!maigret trump123 trumpfan makeamericagreatagain"]},{"cell_type":"markdown","metadata":{"id":"iuwPhT8JsWsi"},"source":["## **Generate reports** 📄\n","---\n","```maigret <username> --<output_type_1> --<output_type_2> ... --<output_type_8>```\n","\n","Possible output type flags: ```--pdf```, ```--html```, ```--xmind```, ```--csv```, ```--txt```, ```--json```, ```--ndjson```\n","\n","```--ndjson``` is the only unique option and tells maigret to generate an individual JSON report for each of the multiple usernames that are simulataneously queried.\n","\n","If using Google Colab, a folder called \"reports\" will automatically be created and the reports will be available in this folder.\n","\n","To see how to generate reports, run the following code which will generate a .pdf, .html, and .csv version of your username dossier."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"N_NsUlcDsfdc"},"outputs":[],"source":["!maigret trump123 --pdf --html --csv"]},{"cell_type":"markdown","metadata":{"id":"jTl6OYrzcJfA"},"source":["### Specify file location\n","_____\n","\n","By default, any file-based reports that you generate will be stored in a \"reports\" folder. This folder will be created even if it doesn't already exist.\n","\n","To output your file at a specific location in your folder, enter your command in the following format:\n","\n","```maigret <username_1> --<output_type> --folderoutput <file_path>```\n","\n","Here is an example:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Ua70mOlYqpG-"},"outputs":[],"source":["!maigret trump123 --pdf --folderoutput cool_files/pdfs"]},{"cell_type":"markdown","metadata":{"id":"2laAxl_qu5jo"},"source":["## **Filter searched websites via tags**\n","---\n","Generally, your command will take one of the following formats:\n","1. Query websites related to one tag for a username: `maigret <username> --tags <tag>`\n","2. Query websites related to multiple tags for a username: `maigret <username> --tags <tag_1>,<tag_2>,...,<tag_n>`\n","\n","### **See below for more information on the 3 categories of tags available** 👇\n","\n","### Country Code `tags` 🏁\n","---\n","You can use tags to focus your search on sites in a specific language or with a userbase in a specific country.\n","\n","Here's an example of how to query sites either written in Japanese or related to Japan for information on a username:\n","\n","\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"E64WWUDdo_s6"},"outputs":[],"source":["!maigret trump123 --tags jp"]},{"cell_type":"markdown","metadata":{"id":"1ey2S2CDphvQ"},"source":["Optionally, you could also query multiple country codes.\n","The following is an example of how to query both Japan- and US-linked sites."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"bijEPUDSpxY1"},"outputs":[],"source":["!maigret trump123 --tags jp,us"]},{"cell_type":"markdown","metadata":{"id":"eICZ8kdsprlq"},"source":["Scroll down to the \"Current Codes\" section on [this page](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) for all the possible country codes."]},{"cell_type":"markdown","metadata":{"id":"O30Czab8pvKC"},"source":["### Site Engine `tags`\n","---\n","\n","These are mostly tags for online community forums (they mainly correspond to sites either in Russia or with a largely Russian-speaking userbase)."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"nWn2bu7RqCjx"},"outputs":[],"source":["!maigret trump123 --tags uCoz"]},{"cell_type":"markdown","metadata":{"id":"sQSl2nz1qKTe"},"source":["**Some common site engines** include: uCoz, vBulletin, phpBB, XenForo, and Discourse."]},{"cell_type":"markdown","metadata":{"id":"TvzqVTj9KLTy"},"source":["\n","### Sites Subject/Genre `tags`\n","---\n","You can use tags to focus your search on sites that correspond to a specific topic, genre, or interest.\n","\n","Here's an example of how to query sites related to dating and politics:\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"j5gsFd4Kreb8"},"outputs":[],"source":["!maigret trump123 --tags dating,politics"]},{"cell_type":"markdown","metadata":{"id":"oI6u2KiMlxVB"},"source":["**Other possible subject tags** include: `forum, gaming, dating, politics, porn, photo, sharing, blog, tech, freelance, video, news, art, music, finance, discussion, coding, messaging, writing, geosocial, networking, freelance, maps, documents`"]},{"cell_type":"markdown","metadata":{"id":"KhY1q1RAt1dt"},"source":["### Combining different types of tags\n","------\n","Below an example of how we can query dating sites in Japan for our username. The format is the same as previous examples of using tags, but in this example we are mixing different types of tags:\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"KBVXj3VMuPyN"},"outputs":[],"source":["!maigret trump123 --tags dating,jp"]},{"cell_type":"markdown","metadata":{"id":"EiWtQfmqrbgI"},"source":["\n","## **Customizing the scope of your search** 🔍\n","-----\n","By default, searches are executed on the top 500 sites on [this list](https://github.com/soxoj/maigret/blob/main/sites.md)."]},{"cell_type":"markdown","metadata":{"id":"aV-Y57UrU5Sr"},"source":["### Search all possible sites (3000)\n","_____\n","You can instruct Maigret to search all the sites on [this list](https://github.com/soxoj/maigret/blob/main/sites.md).\n","\n","Here's an example of how to to do that (using \"-a\"):"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"40_uvgdOU9C0"},"outputs":[],"source":["!maigret trump2016 -a"]},{"cell_type":"markdown","metadata":{"id":"aGar_pAzU-VZ"},"source":["### Search an individual site\n","_____\n","Using the command below, you can use maigret to gather information on a username from one website. This website should be from one of the 3000 websites in the list linked in the 2 sections above."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"eamz3Ts9Wgqz"},"outputs":[],"source":["!maigret trump2016 --site Facebook"]},{"cell_type":"markdown","metadata":{"id":"g0Uoq6eJXKSK"},"source":["### Analyze an individual profile URL\n","____\n","With this functionally, you can paste the URL of a user profile and execute a search based on the information in the profile.\n","\n","Enter your command in the following format:\n","\n","```maigret --parse <profile_url>```\n","\n","Maigret will parse this profile to find a username and then use that username to query 500 of the top sites in the list referenced above.\n","\n","Below is an example where maigret analyzes and executes a search based on a Steam profile:"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"C40LauBvXQZX"},"outputs":[],"source":["!maigret --parse https://steamcommunity.com/profiles/76561199113454789"]},{"cell_type":"markdown","metadata":{"id":"wzjfspYWYige"},"source":["# **FAQ** 📃"]},{"cell_type":"markdown","metadata":{"id":"AMABHT6tVK_8"},"source":["### **How can I deal with rate limits?**\n","----\n","If using maigret on your own computer:\n","- You can connect to a VPN and then make a request - this effectively changes the IP address from which you are making your request to a given website.\n","- You can also run the application on a different computer or WiFi connection, public spaces or campuses may not be so strongly rate-limited.\n","\n","If using maigret in a Jupyter Notebook, it really depends on the machine hosting your Jupyter Notebook and whether that machine can make a request using a VPN. If not, you may simply have to wait for a time period determined by the website that's rate limiting you.\n","\n","### **How can I combine the --parameters?**\n","----\n","\n","You can combine tags and any related parameters in any order you want. The usernames come first and then all the other --parameters should follow.\n","\n","Here's an example of running a query on multiple usernames against websites in Japan, and exporting the results to a .csv file."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"k6SzL3_7xFd7"},"outputs":[],"source":["!maigret trump123 trump_lover --tags jp --csv"]},{"cell_type":"markdown","metadata":{"id":"s6xwFqgG57g7"},"source":["Here's an example of running a query on all possible sites (3000) for a username and outputting the result to both a pdf and csv."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"eO6rXkhK6NsR"},"outputs":[],"source":["!maigret trump123 -a --csv --pdf"]}],"metadata":{"colab":{"provenance":[]},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"}},"nbformat":4,"nbformat_minor":0}