{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Project 3 - Search-Based Web Fuzzer " ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import fuzzingbook_utils" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introduction\n", "\n", "Fuzzing web applications can be challenging. A simple web form requires a specific input format for each field, this format needs to be fulfilled for a valid web interaction (e.g. an email field is expected to fulfill the following regex pattern `r'^([a-z0-9_\\.-]+)@([\\da-z\\.-]+)\\.([a-z\\.]{2,6})$')`.\n", "\n", "In this project, we demonstrate how to apply a search-based algorithm to generate specific test inputs for web interfaces. The generated inputs should exercise the web application by fulfilling the input validation requirements of all fields required to reach specific web pages (both normal and error pages). \n", "\n", "The task is to employ a genetic algorithm (GA) for fuzzing, your implementation is expected to generate inputs that match the expected pattern or constraints of the input field (e.g. email fuzzingbook@gmail.com fulfils the expected email regex pattern `r'^([a-z0-9_\\.-]+)@([\\da-z\\.-]+)\\.([a-z\\.]{2,6})$')`. This is to be achieved by searching the input space starting from an initially (random) input. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from WebFuzzer import init_db, ORDERS_DB, SimpleHTTPRequestHandler \n", "from http.server import HTTPServer, HTTPStatus " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Objective\n", "\n", "The goal of this project is to _implement a search-based fuzzing algorithm that generates specifically formatted inputs which fulfil the input validation requirements of a web form_. \n", "\n", "You will apply techniques learned in the lecture ([SBST](SBST.ipynb) and [WebFuzzer](WebFuzzer.ipynb)) to automatically fill web forms by producing inputs that improve the reachability of web pages. \n", "\n", "The goal of your Fuzzer is to cleverly search the input space until the input validation format of the input field is achieved. Consequently, fulfilling (or not fulfilling) such input validation schemes achieves the reachability of specific web pages and improves the coverage of the site map. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Web Application\n", "\n", "We create a (HTML only) web application for placing an order of items, similar to the order form in the [WebFuzzer](WebFuzzer.ipynb) lecture.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Input Validation\n", "\n", "We create a `ProjectHTTPRequestHandler` class that provides a web order form with input validation schemes for all input formats. \n", "\n", "First, we create a sample set of regex for input validation of each input field in our order form. \n", "\n", "__Note that your implementation shall be evaluated with a _secret set of regex_ other than the ones provided below.__\n", "\n", "For some input fields, we have provided other sample regexes you may use to test your implementation, other regexes are provided to inform you of the nature of the regexes which may be used to evaluate your final solution. \n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "itemList = r'^(tshirt|drill|lockset)$'\n", "name_regex = [r'^[A-Z][a-z]+ [A-Z][a-z]+ [A-Z][a-z]+$', r'^[A-Z][a-z]+ [A-Z]\\. [A-Z][a-z]+$', \\\n", " r'^[A-Z][a-z]+ [A-Z][a-z]+$', r'^[a-z]+ [a-z]+$']\n", "email_regex = [r'^([a-z0-9_\\.-]+)@([\\da-z\\.-]+)\\.([a-z\\.]{2,6})$', r'^([a-z0-9_\\.-]+)@cispa\\.saarland$', \\\n", " r'^([a-z0-9_\\.-]+)@gmail\\.com$', r'^([a-z0-9_\\.-]+)@hotmail\\.co\\.uk$']\n", "zip_regex = [r'^\\d{5}$', r'^\\d{3}-\\d{4}$', r'^\\d{5}([ \\-]\\d{4})?$', \\\n", " r'^[ABCEGHJKLMNPRSTVXY]\\d[ABCEGHJ-NPRSTV-Z][ ]?\\d[ABCEGHJ-NPRSTV-Z]\\d$']\n", "city_regex = [r'^[A-Z]\\w{2,9}$', r'^[a-zA-Z]+(?:[\\s-][a-zA-Z]+)*$', r'^\\p{Lu}\\p{L}*(?:[\\s-]\\p{Lu}\\p{L}*)*$', r'^\\w{2,12}$']\n", "terms_regex = r'^on$'\n", "\n", "list_regex = [itemList, name_regex[0], email_regex[0], city_regex[0], zip_regex[0], terms_regex]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We define diagnostic error messages for each error page." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "error_msg_list = [\"INCOMPLETE INPUT: \\n All fields have to be filled \",\\\n", " \"INVALID INPUT: \\n Item is not in item list\",\\\n", " \"INVALID INPUT: \\n Name does not match expected pattern :\\n \",\\\n", " \"INVALID INPUT: \\n Email does not match expected pattern :\\n \",\\\n", " \"INVALID INPUT: \\n City does not match expected pattern :\\n \",\\\n", " \"INVALID INPUT: \\n Zip does not match expected pattern :\\n \",\\\n", " \"INVALID INPUT: \\n Terms and Conditions have to be checked \"\n", " ]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We override the `handle_order()` method, to validate the values of each input field using the pre-defined regex patterns above. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We use the [regex parser](https://pypi.org/project/regex/), specifically version 2.5.23, instead of Python's native `re`, because `re` does not support the `\\p{}` syntax. Confirm regex version using: " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pip install regex \n", "import regex\n", "print(regex.__version__)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "#import re\n", "\n", "class ProjectHTTPRequestHandler(SimpleHTTPRequestHandler):\n", " def handle_order(self): \n", " values = self.get_field_values()\n", "\n", " if len(values) < 6:\n", " error_type = error_msg_list[0]\n", " self.internal_server_error(error_type)\n", " \n", " elif regex.match(list_regex[0], values[\"item\"]) and regex.match(list_regex[1], values[\"name\"]) and \\\n", " regex.match(list_regex[2], values[\"email\"]) and regex.match(list_regex[4], values[\"zip\"]) and \\\n", " regex.match(list_regex[3], values[\"city\"]) and regex.match(list_regex[5], values[\"terms\"]):\n", " self.store_order(values)\n", " self.send_order_received(values)\n", "\n", " elif not regex.match(list_regex[0], values[\"item\"]):\n", " error_type = error_msg_list[1]\n", " self.internal_server_error(error_type) \n", " \n", " elif not regex.match(list_regex[1], values[\"name\"]):\n", " error_type = error_msg_list[2] + name_regex[0] \n", " self.internal_server_error(error_type)\n", " \n", " elif not regex.match(list_regex[2], values[\"email\"]):\n", " error_type = error_msg_list[3] + email_regex[0]\n", " self.internal_server_error(error_type)\n", " \n", " elif not regex.match(list_regex[3], values[\"city\"]):\n", " error_type = error_msg_list[4] + city_regex[0]\n", " self.internal_server_error(error_type)\n", " \n", " elif not regex.match(list_regex[4], values[\"zip\"]):\n", " error_type = error_msg_list[5] + zip_regex[0]\n", " self.internal_server_error(error_type)\n", " \n", " elif not regex.match(list_regex[5], values[\"terms\"]):\n", " error_type = error_msg_list[6]\n", " self.internal_server_error(error_type)\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Internal Errors\n", "For diagnostic (and identification) purposes, the internal server error pages include the error message indicating the input validation scheme that failed." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "HTML_INTERNAL_SERVER_ERROR = \"\"\"\n", "\n", "
\n", " Internal Server Error\n", "

\n", " The server has encountered an internal error. Go to our order form.\n", " 

{error_message}
\n", "

\n", "
\n", "\n", " \"\"\"" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "import sys\n", "import traceback\n", "\n", "class ProjectHTTPRequestHandler(ProjectHTTPRequestHandler):\n", " def internal_server_error(self, error_type=None):\n", " self.send_response(HTTPStatus.OK, \"Internal Error\")\n", "\n", " self.send_header(\"Content-type\", \"text/html\")\n", " self.end_headers()\n", "\n", " exc = traceback.format_exc()\n", " self.log_message(\"%s\", exc.strip())\n", "\n", " message = HTML_INTERNAL_SERVER_ERROR.format(error_message=error_type)\n", " self.wfile.write(message.encode(\"utf8\"))" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "
\n", " Internal Server Error\n", "

\n", " The server has encountered an internal error. Go to our order form.\n", " 

{error_message}
\n", "

\n", "
\n", "\n", " " ], "text/plain": [ "" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from fuzzingbook_utils import HTML \n", "HTML(HTML_INTERNAL_SERVER_ERROR)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Storing Orders" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Similar to the [WebFuzzer](WebFuzzer.ipynb) lecture, we store orders using a *database* stored in the file `orders.db`." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "FUZZINGBOOK_SWAG = {\n", " \"tshirt\": \"One FuzzingBook T-Shirt\",\n", " \"drill\": \"One FuzzingBook Rotary Hammer\",\n", " \"lockset\": \"One FuzzingBook Lock Set\"\n", "}" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "import sqlite3, os\n", "\n", "class ProjectHTTPRequestHandler(ProjectHTTPRequestHandler):\n", " def store_order(self, values):\n", " db = sqlite3.connect(ORDERS_DB)\n", " db.execute(\"INSERT INTO orders VALUES (?, ?, ?, ?, ?, ?)\",\n", " (values['item'], values['name'], values['email'], values['city'], values['zip'], values['terms']))\n", " db.commit()\n", "\n", " def send_order_received(self, values):\n", " values[\"item_name\"] = FUZZINGBOOK_SWAG[values[\"item\"]]\n", " confirmation = HTML_ORDER_RECEIVED.format(**values).encode(\"utf8\")\n", "\n", " self.send_response(HTTPStatus.OK, \"Order received\")\n", " self.send_header(\"Content-type\", \"text/html\")\n", " self.end_headers()\n", " self.wfile.write(confirmation)\n", " " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "def init_db():\n", " if os.path.exists(ORDERS_DB):\n", " os.remove(ORDERS_DB)\n", "\n", " db_connection = sqlite3.connect(ORDERS_DB)\n", " db_connection.execute(\"DROP TABLE IF EXISTS orders\")\n", " db_connection.execute(\"CREATE TABLE orders (item text, name text, email text, city text, zip text, terms text)\")\n", " db_connection.commit()\n", "\n", " return db_connection\n", "\n", "db = init_db()" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[]\n" ] } ], "source": [ "print(db.execute(\"SELECT * FROM orders\").fetchall())\n", "\n", "db.execute(\"INSERT INTO orders \" +\n", " \"VALUES ('lockset', 'Jane Lee Doe', 'jane@doe.com', 'Saarbrucken', '66125', 'on')\")\n", "db.commit()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[('lockset', 'Jane Lee Doe', 'jane@doe.com', 'Saarbrucken', '66125', 'on')]\n" ] } ], "source": [ "print(db.execute(\"SELECT * FROM orders\").fetchall())\n", "\n", "\n", "db.execute(\"DELETE FROM orders WHERE name = 'Walter White'\")\n", "db.commit()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[('lockset', 'Jane Lee Doe', 'jane@doe.com', 'Saarbrucken', '66125', 'on')]\n" ] } ], "source": [ "print(db.execute(\"SELECT * FROM orders\").fetchall())" ] }, { "cell_type": "markdown", "metadata": { "toc-hr-collapsed": false }, "source": [ "### Order Confirmation\n", "\n", "Once we have gotten an order, we show a confirmation page, which is instantiated with the customer information submitted before." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "button": false, "new_sheet": false, "run_control": { "read_only": false }, "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "HTML_ORDER_RECEIVED = \"\"\"\n", "\n", "
\n", " Thank you for your Fuzzingbook Order!\n", "

\n", " We will send {item_name} to {name} in {city}, {zip}
\n", " A confirmation mail will be sent to {email}.\n", "

\n", "

\n", " Want more swag? Use our order form!\n", "

\n", "
\n", "\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Logging \n", "\n", "Similar to the [WebFuzzer](WebFuzzer.ipynb) lecture, we provide a logging infrastructure for debugging purposes. We implement the following:\n", "- `print_httpd_messages()` prints all messages accumulated in the queue\n", "- `clear_httpd_messages()` discards all pending messages\n", "- `log_message()` store messages in the queue " ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "from multiprocessing import Queue, Process\n", "from fuzzingbook_utils import rich_output, terminal_escape, unicode_escape, HTML\n", "\n", "HTTPD_MESSAGE_QUEUE = Queue()\n", "\n", "def display_httpd_message(message):\n", " if rich_output():\n", " display(\n", " HTML(\n", " '
' +\n",
    "                message +\n",
    "                \"
\"))\n", " else:\n", " print(terminal_escape(message))\n", "\n", "def print_httpd_messages():\n", " while not HTTPD_MESSAGE_QUEUE.empty():\n", " message = HTTPD_MESSAGE_QUEUE.get()\n", " display_httpd_message(message)\n", "\n", "def clear_httpd_messages():\n", " while not HTTPD_MESSAGE_QUEUE.empty():\n", " HTTPD_MESSAGE_QUEUE.get()\n", "\n", "class ProjectHTTPRequestHandler(ProjectHTTPRequestHandler):\n", " def log_message(self, format, *args):\n", " message = (\"%s - - [%s] %s\\n\" %\n", " (self.address_string(),\n", " self.log_date_time_string(),\n", " format % args))\n", " HTTPD_MESSAGE_QUEUE.put(message)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Extend `webbrowser()` method to prints log messages produced by the server:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "from Carver import webbrowser as simple_webbrowser\n", "\n", "def webbrowser(url, mute=False):\n", " try:\n", " contents = simple_webbrowser(url)\n", " finally:\n", " if not mute:\n", " print_httpd_messages()\n", " else:\n", " clear_httpd_messages()\n", "\n", " return contents" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Running the Web Application\n", "Similar to the [WebFuzzer](WebFuzzer.ipynb) lecture, to run the web application we implement the following:\n", "\n", "* `run_httpd_forever()`\n", "* `start_httpd()`\n", "* `print_url()`\n" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
http://127.0.0.1:8800
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def run_httpd_forever(handler_class):\n", " host = \"127.0.0.1\" # localhost IP\n", " for port in range(8800, 9000):\n", " httpd_address = (host, port)\n", "\n", " try:\n", " httpd = HTTPServer(httpd_address, handler_class)\n", " break\n", " except OSError:\n", " continue\n", "\n", " httpd_url = \"http://\" + host + \":\" + repr(port)\n", " HTTPD_MESSAGE_QUEUE.put(httpd_url)\n", " httpd.serve_forever()\n", "\n", "def start_httpd(handler_class=ProjectHTTPRequestHandler):\n", " clear_httpd_messages()\n", "\n", " httpd_process = Process(target=run_httpd_forever, args=(handler_class,))\n", " httpd_process.start()\n", "\n", " httpd_url = HTTPD_MESSAGE_QUEUE.get()\n", " return httpd_process, httpd_url\n", "\n", "httpd_process, httpd_url = start_httpd(ProjectHTTPRequestHandler)\n", "\n", "def print_url(url):\n", " if rich_output():\n", " display(HTML('
%s
' % (url, url)))\n", " else:\n", " print(terminal_escape(url))\n", "\n", "print_url(httpd_url)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Note**: The above URL only works if you are running the Jupyter notebook server on the local host." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Testing the Web Application\n", "### Input Format\n", "\n", "When the user clicks `Submit` on the order form, the Web browser creates and retrieves a URL of the form:\n", "\n", "```\n", "/order?item=value_1&name=value_2&email=value_3&city=value_4&zip=value_5&terms=value_6\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We test standard behavior:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:14] \"GET /order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=on HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", "
\n", " Thank you for your Fuzzingbook Order!\n", "

\n", " We will send One FuzzingBook T-Shirt to Jane Lee Doe in Seattle, 98104
\n", " A confirmation mail will be sent to doe@example.com.\n", "

\n", "

\n", " Want more swag? Use our order form!\n", "

\n", "
\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "standard_order = \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=on\"\n", "contents = webbrowser(httpd_url + standard_order)\n", "HTML(contents)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "assert contents.find(\"Thank you\") and contents.find(\"We will send\") and contents.find(\"A confirmation mail will be sent to\") > 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We test erroneous behavior - terms and conditions not checked:" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:14] \"GET /order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=off HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:14] NoneType: None\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", "
\n", " Internal Server Error\n", "

\n", " The server has encountered an internal error. Go to our order form.\n", " 

INVALID INPUT: \n",
       "  Terms and Conditions have to be checked
\n", "

\n", "
\n", "\n", " " ], "text/plain": [ "" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "error_order = \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=off\"\n", "contents = webbrowser(httpd_url + error_order)\n", "HTML(contents)" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "assert contents.find(\"Internal Server Error\") and contents.find(error_msg_list[6]) > 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We test incomplete order behavior:" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:14] \"GET /order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104 HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:14] NoneType: None\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", "
\n", " Internal Server Error\n", "

\n", " The server has encountered an internal error. Go to our order form.\n", " 

INCOMPLETE INPUT: \n",
       "  All fields have to be filled
\n", "

\n", "
\n", "\n", " " ], "text/plain": [ "" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "incomplete_order = \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104\"\n", "contents = webbrowser(httpd_url + incomplete_order)\n", "HTML(contents)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "assert contents.find(\"Internal Server Error\") and contents.find(error_msg_list[0]) > 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To test for another input validaion regex (e.g. the second regex pattern for each field - index 1), use:" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
http://127.0.0.1:8801
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:14] \"GET /order?item=tshirt&name=Jane+L.+Doe&email=doe%40cispa.saarland&city=Seattle&zip=984-1234&terms=on HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "list_regex = [itemList, name_regex[1], email_regex[1], city_regex[1], zip_regex[1], terms_regex]\n", "httpd_process, httpd_url = start_httpd(ProjectHTTPRequestHandler)\n", "print_url(httpd_url)\n", "new_regex_standard_order = \"/order?item=tshirt&name=Jane+L.+Doe&email=doe%40cispa.saarland&city=Seattle&zip=984-1234&terms=on\"\n", "contents = webbrowser(httpd_url + new_regex_standard_order)\n", "HTML(contents)\n", "assert contents.find(\"Thank you\") and contents.find(\"We will send\") and contents.find(\"A confirmation mail will be sent to\") > 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, we reset the server to use the first set of regex for input validation (index 0):" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
http://127.0.0.1:8802
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "list_regex = [itemList, name_regex[0], email_regex[0], city_regex[0], zip_regex[0], terms_regex]\n", "httpd_process, httpd_url = start_httpd(ProjectHTTPRequestHandler)\n", "print_url(httpd_url)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Test for the first set of regex again:" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:15] \"GET /order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=on HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "contents = webbrowser(httpd_url + standard_order)\n", "assert contents.find(\"Thank you\") and contents.find(\"We will send\") and contents.find(\"A confirmation mail will be sent to\") > 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We test erroneous behavior - invalid city:" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:15] \"GET /order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=1&zip=98104&terms=on HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:15] NoneType: None\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "wrong_city_order = \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=1&zip=98104&terms=on\"\n", "contents = webbrowser(httpd_url + wrong_city_order)\n", "HTML(contents)\n", "assert contents.find(\"Internal Server Error\") and contents.find(error_msg_list[4]) > 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We test erroneous behavior - invalid zip code:" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:15] \"GET /order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=104&terms=on HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:15] NoneType: None\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "wrong_zip_order = \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=104&terms=on\"\n", "contents = webbrowser(httpd_url + wrong_zip_order)\n", "HTML(contents)\n", "assert contents.find(\"Internal Server Error\") and contents.find(error_msg_list[5]) > 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Obtaining Coverage\n", "\n", "We define the `target_list` for each web page accordingly:" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "target_list:\n", " ISE_Name = Internal Server Error Web Page for \"Name\" field error, analogously for ISE_Item, ISE_City, etc..\n", " ISE_INC = Internal Server Error Web Page for \"Incomplete Input\" \n", " PNF = Page Not Found Web Page\n", " CONF = CONFIRMATION_PAGE\n", "\"\"\"\n", "\n", "#target_list = [\"ISE_INC\", \"ISE_Item\", \"ISE_Name\", \"ISE_Email\", \"ISE_Zip\", \"ISE_City\",\"ISE_Terms\", \"CONF\", \"PNF\"]\n", "target_list = [\"ISE_INC\", \"ISE_Item\", \"ISE_Name\", \"ISE_Email\", \"ISE_City\",\"ISE_Zip\", \"ISE_Terms\", \"CONF\", \"PNF\"]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To obtain the overall coverage achieved by the GA's resulting population, we implement the following web reachability function. \n", "\n", "For each target, we compute:\n", "* _validity_ score: the number of valid inputs in the generated population that reach the target, and\n", "* _reachability_ score: we score the number of inputs that reach the each page. \n", "\n", "The reachability scores for the number of inputs reaching the target is based on the perceived difficulty to reach the page:\n", "* 2 points for each input reaching an error page\n", "* 5 points for each input reaching the confirmation page\n", "* 1 point for each input reaching the page not found page\n", "\n", "**Note**: we expect each run of your genetic algorithm to produce 10 inputs maximum." ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "def get_web_reachability(orders, target):\n", " num_reached_pages = dict() \n", " reachability, validity = 0, 0\n", " conf_msg = [\"Thank you\", \"We will send\", \"A confirmation mail will be sent to\"]\n", " page_not_found = \"This page does not exist\"\n", " \n", " for order in orders:\n", " try:\n", " clear_httpd_messages()\n", " contents = webbrowser(httpd_url + order, mute=True)\n", " HTML(contents)\n", " \n", " # obtain 2 reachability points for each input reaching an error page\n", " if contents.find(\"Internal Server Error\") > 0:\n", " if target_list.index(target) <= 6:\n", " msg = error_msg_list[target_list.index(target)]\n", " if contents.find(msg) > 0: \n", " if not msg in num_reached_pages:\n", " num_reached_pages[msg] = 2\n", " else:\n", " num_reached_pages[msg] += 2\n", " validity += 1\n", "\n", " # obtain 5 reachability points for each input reaching confirmation page\n", " if target == target_list[7] and contents.find(conf_msg[0]) > 0 and contents.find(conf_msg[1]) > 0 \\\n", " and contents.find(conf_msg[2]) > 0:\n", " if not conf_msg[0] in num_reached_pages:\n", " num_reached_pages[conf_msg[0]] = 5\n", " else:\n", " num_reached_pages[conf_msg[0]] += 5\n", " validity += 1\n", "\n", " # obtain 1 reachability point for each input reaching a non-existent page\n", " if target == target_list[8] and contents.find(page_not_found) > 0:\n", " if not page_not_found in num_reached_pages:\n", " num_reached_pages[page_not_found] = 1\n", " else:\n", " num_reached_pages[page_not_found] += 1\n", " validity += 1\n", " \n", " except Exception as e:\n", " e.__traceback__ = None\n", " traceback.clear_frames(e.__traceback__)\n", " pass\n", " \n", " \n", " if len(num_reached_pages) > 0:\n", " reachability = sum(num_reached_pages.values())\n", " return reachability, validity" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We test the `get_web_reachability` function for the `confirmation` page target with a sample list of test order URLs. " ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "test_orders = [\"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=off\",\\\n", " \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=on\",\\\n", " \"/order\", \"/none\",\\\n", " \"/order?item=lockset&name=Jane+lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=on\",\\\n", " \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40cispa.saarland&city=Seattle&zip=98104&terms=on\",\\\n", " \"/order?item=book&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=on\",\\\n", " \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=98104&terms=on\",\\\n", " \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Seattle&zip=498104&terms=on\",\\\n", " \"/order?item=tshirt&name=Jane+Lee+Doe&email=doe%40example.com&city=Neunkirchen&zip=98104&terms=on\",\\\n", " ]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Test for the reachability of confirmation page (\"CONF\") using the above order inputs:" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(10, 2)" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_web_reachability(test_orders, \"CONF\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Search Based Fuzzing\n", "\n", "We demonstrate how a specific input format can be generated with a simple search-based algorithm and a naive fitness function. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We assume an input url with an invalid name format error:" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:15] \"GET /order?item=tshirt&name=Jane+Lee+Doe1&email=doe%40example.com&city=Seattle&zip=98104&terms=on HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:15] NoneType: None\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import random, string\n", "invalid_name_order = \"/order?item=tshirt&name=Jane+Lee+Doe1&email=doe%40example.com&city=Seattle&zip=98104&terms=on\"\n", "contents = webbrowser(httpd_url + invalid_name_order)\n", "HTML(contents)\n", "assert contents.find(\"Internal Server Error\") and contents.find(error_msg_list[2]) > 0" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "import urllib.parse\n", "test_name = urllib.parse.parse_qs(invalid_name_order, keep_blank_values=True)[\"name\"][0] " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We implement a naive distance function for \"name\" input field:" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [], "source": [ "def naive_calculate_name_distance(name):\n", " name_regex = r'[A-Z][a-z]+ [A-Z][a-z]+ [A-Z][a-z]+'\n", " res = 1\n", " if len(name.split()) == 3 and check_ascii_letters(name) and \\\n", " check_word_size(name) and name.istitle():\n", " res = 0\n", " else:\n", " if len(name.split()) == 3:\n", " res -= 0.4\n", " if check_ascii_letters(name):\n", " res -= 0.35\n", " if check_word_size(name):\n", " res -= 0.15\n", " if name.istitle():\n", " res -= 0.10\n", " return res \n", "\n", "def check_ascii_letters(name):\n", " res = True\n", " for char in name.replace(\" \", \"\"):\n", " if not (char in string.ascii_letters):\n", " res = False\n", " break\n", " return res\n", "\n", "def check_word_size(name):\n", " res = True\n", " for word in name.split():\n", " if len(word) < 2:\n", " res = False\n", " break\n", " return res" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We implement a naive fitness function for the name input. \n" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [], "source": [ "def get_naive_fitness(name):\n", " global distance\n", " distance = naive_calculate_name_distance(name)\n", " fitness = distance\n", " return fitness" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We implement a simple mutation function `flip_random_character` for the name input. " ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [], "source": [ "def flip_random_character(s):\n", " pos = random.randint(0, len(s) - 1)\n", " new_c = chr(random.randrange(65, 130))\n", " return s[:pos] + new_c + s[pos + 1:]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We implement a simple search-based algorithm -`randomized_hillclimber()`, to run for a maximum of 5000 iterations :" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "def randomized_hillclimber(name):\n", " fitness = get_naive_fitness(name)\n", " print(\"Initial value: %s at fitness %.4f\" % (name, fitness))\n", " iterations = 0\n", "\n", " while fitness > 0 and iterations < 5000:\n", " iterations += 1\n", " \n", " mutated = flip_random_character(name)\n", " new_fitness = get_naive_fitness(mutated)\n", " if new_fitness <= fitness:\n", " name = mutated\n", " fitness = new_fitness\n", "\n", " if iterations < 5000:\n", " print(\"Optimum at %s after %d iterations\" % (name, iterations))\n", " else:\n", " print(\"# iterations is: %d, stop!!!\" % (iterations))\n", " return name, iterations" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Initial value: Jane Lee Doe1 at fitness 0.3500\n", "Optimum at Rhvf Ctz Yvun after 2544 iterations\n" ] } ], "source": [ "for i in range(0, 5):\n", " new_name, no_iterations = randomized_hillclimber(test_name) \n", " if no_iterations < 5000:\n", " new_name = new_name.replace(\" \", \"+\")\n", " new_order = \"/order?item=tshirt&name=\" + new_name + \"&email=doe%40example.com&city=Seattle&zip=98104&terms=on\"\n", " new_order\n", " break\n", " new_name = new_name.replace(\" \", \"+\")\n", " new_order = \"/order?item=tshirt&name=\" + new_name + \"&email=doe%40example.com&city=Seattle&zip=98104&terms=on\"\n", " new_order" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
127.0.0.1 - - [24/Feb/2019 22:01:15] \"GET /order?item=tshirt&name=Rhvf+Ctz+Yvun&email=doe%40example.com&city=Seattle&zip=98104&terms=on HTTP/1.1\" 200 -\n",
       "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "clear_httpd_messages()\n", "contents = webbrowser(httpd_url + new_order)\n", "HTML(contents)\n", "assert contents.find(\"Thank you\") and contents.find(\"We will send\") and contents.find(\"A confirmation mail will be sent to\") > 0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Genetic Algorithm" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Genetic Algorithm template\n", "\n", "The basic template of the genetic algorithm is based on the [SBST](SBST.ipynb) from the lecture.\n", "\n", "This template uses your implemented fitness, selection, mutation and crossover functions for the population of input urls. " ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [], "source": [ "def evaluate_population(population, target, list_regex):\n", " fitness = [get_fitness(x, target, list_regex) for x in population]\n", " return list(zip(population, fitness))" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [], "source": [ "def terminal_repr(s):\n", " return terminal_escape(repr(s))" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [], "source": [ "import time\n", "\n", "def genetic_algorithm(population, target, list_regex):\n", " # Generate and evaluate initial population\n", " generation = 0\n", " fitness = evaluate_population(population, target, list_regex)\n", " best = min(fitness, key=lambda item: item[1])\n", " best_individual = best[0]\n", " best_fitness = best[1]\n", " #print(\"Best fitness of initial population: %s - %.10f\" %\n", " # (terminal_repr(best_individual), best_fitness))\n", "\n", " start_time = time.time()\n", " time_spent = 0\n", " \n", " # Stop when optimum found, or we run out of patience, i.e. 5000 iterations or 5 hours max\n", " while best_fitness > 0 and generation < 5000 and (time_spent <= 18000):\n", "\n", " # The next generation will have the same size as the current one\n", " new_population = []\n", " while len(new_population) < len(population):\n", " # Selection\n", " offspring1 = selection(fitness, 10)\n", " offspring2 = selection(fitness, 10)\n", "\n", " # Crossover\n", " if random.random() < 0.7:\n", " (offspring1, offspring2) = crossover(offspring1, offspring2)\n", "\n", " # Mutation\n", " offspring1 = mutate(offspring1)\n", " offspring2 = mutate(offspring2)\n", "\n", " new_population.append(offspring1)\n", " new_population.append(offspring2)\n", "\n", " # Once full, the new population replaces the old one\n", " generation += 1\n", " population = new_population\n", " fitness = evaluate_population(population, target, list_regex)\n", "\n", " best = min(fitness, key=lambda item: item[1])\n", " best_individual = best[0]\n", " best_fitness = best[1]\n", " \n", " \n", " end_time = time.time()\n", " time_spent = end_time - start_time\n", "\n", " #print(\n", " # \"Best fitness at generation %d: %s - %.8f\" %\n", " # (generation, terminal_repr(best_individual), best_fitness))\n", "\n", " #print(\n", " # \"Best individual: %s, fitness %.10f\" %\n", " # (terminal_repr(best_individual), best_fitness))\n", " return population, generation" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Your Tasks\n", "For each input, you are epxected to produce a set of urls to reach a targeted web page, i.e. generate inputs that will fulfill the input validation requirements necessary to reach a specific web page. Overall, all web pages will be targets, i.e. both error and normal pages. \n", "\n", "Your task is to implement your own custom selection, fitness and mutation functions for the genetic algorithm, in order to fulfill the input validation and web reachabilty requirements." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tips\n", "\n", "* Your fitness function should target a specific web page,e.g. a specific error page like an invalid name error page. \n", "* For input (fragment) distance consider using a (fuzzy/approximate) regex matching distance of the input to the expected regex.\n", "* You are allowed to use the `fuzzingbook` module, any of the [inbuilt python modules](https://docs.python.org/3/py-modindex.html) and the [regex module](https://pypi.org/project/regex/). \n", "* __Do not import external modules other than the afore-mentioned modules, and do not use (parts of) open source code__.\n", "* You can develop any type of mutation, selection and crossover functions.\n", "* Ensure that your implementation accounts for any arbitrary regex and any random initial population of inputs.\n", "* For debugging purposes: unmute the `webbrowser()` to obtain logging information, i.e. set `webbrowser(url, mute=False)` \n", "* Do not implement in any other section except the section below. \n", "* Gracefully handle exceptions and errors resulting from your impelementation.\n", "* __Remember the input validation regex and initial input population could be arbitrary, do not hard code for a specific regex or input__." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Your Code" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In the following section implement the following:\n", "* __Fitness Function(s)__: define a (set of) fitness function for an order URL to validate/invalidate each input field to reach specific pages (target). \n", "\n", "* __Selection Function__: define a selection function for the given generated population of URLs.\n", "\n", "* __Crossover Function__: define a crossover function for two parent input URLs.\n", "\n", "* __Mutate Function__: define a mutate function for our input URL.\n", "\n", "Implement your own fitness, selection and mutation functions for the GA algorithm below:" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "\"\"\"\n", "WRITE YOUR CODE BELOW\n", "\"\"\"\n", "\n", "#e.g. get_fitness(order, target, list_regex), where target = \"CONF\" \n", "\n", "def get_fitness(order, target, list_regex):\n", " # Reset any distance values from previous executions\n", " global distances_true, distances_false\n", " distances_true = {}\n", " distances_false = {}\n", "\n", " # run the webpage with this order\n", " try:\n", " contents = webbrowser(httpd_url + order)\n", " HTML(contents)\n", " except BaseException:\n", " pass\n", "\n", " fitness = 0.0\n", " \n", " \"\"\"\n", " WRITE YOUR CODE HERE \n", " \"\"\"\n", " \n", " return fitness\n", "\n", "def selection(evaluated_population, tournament_size):\n", " \"\"\"\n", " WRITE YOUR CODE HERE \n", " \"\"\"\n", " return winner\n", "\n", "def crossover(parent1, parent2):\n", " \"\"\"\n", " WRITE YOUR CODE HERE \n", " \"\"\"\n", " return (offspring1, offspring2)\n", "\n", "def mutate(chromosome):\n", " \"\"\"\n", " WRITE YOUR CODE HERE \n", " \"\"\"\n", " return mutated" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Evaluation code\n", "\n", "The code in the following section will be used to evaluate your impelementation." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Random Population of Inputs\n", "We randomly generated an `initial_population` of 20 set of inputs for our order web form, with (4) varying levels of difficulty (or garbage). Each populations of input URL contains 10 input URLs for orders.\n", "\n", "__Note that the url path population which will be used for evaluation may be different from those provided as example.__" ] }, { "cell_type": "code", "execution_count": 48, "metadata": { "code_folding": [ 0, 11, 22, 33, 44, 57, 68, 79, 90, 101, 115, 126, 137, 148, 159, 170, 182, 193, 204, 215 ] }, "outputs": [], "source": [ "easy_population_1 = ['/order?item=lockseR&name=RG}rqM`Ak_UTHDdxZtwepzCnmp^MKIF_olhKjBvZ`Ih]`[zQl+_PVkfPqWRhM]x~fbFzKbfwrMFVH^~h+v{ko]lL|CIUVAj{_P_pXpD\\\\SIGpG~Ws&email=qawxodegjvdzrovtd%40cmgjm&city=Wpk^kvTLu&zip=1176&terms=on',\n", " '/order?item=lockseR&name=xjOpvBQETS|uWZ\\\\mxjgRH+Jrf[|^PJzdNv{^+je\\\\Q{ikyikRP&email=vtjbtbqrafmcwtkakv%40kc&city=oF&zip=0023&terms=off',\n", " '/order?item=lockset&name=kLQEm[+nSHuhK{JH{_DWhBZPqW~eO_IhYGUOo\\\\IhEwYRfRQAqeGW{Z+}OtVpB&email=hrumxvplbtrlkbl%40ss&city=rWPMDudM}&zip=68433&terms=on',\n", " '/order?item=drills&name=`[LRcAVCbhPKh\\\\{~}ijb+{sc[SzRl|GXgkduPCrAkETRvOBhjX{UjI+^PKe~S^][&email=dbtmouijqdesfvsxqpydle%40rgz&city=ipF&zip=84627&terms=off',\n", " '/order?item=drills&name=UbRCtoM|NpErry~fuO[Rxua]}Tm~l|LxYX{~sLYb}GVxQsGku+Vo`\\\\LiR|]rk_b~asyaITVb~+VYDvNGMuhgxI^w&email=uahqmmquiab%40xo&city=upuM&zip=12425&terms=off',\n", " '/order?item=tshirts&name=Ft~Uf`|nEXPoOM`IMuQvJ+mX\\\\n^gWwUwf^MRPlCi^ovEjcFg+t\\\\p{jFdAtZuE{sV&email=bv%40bskm&city={gqO\\\\NtzK&zip=5376&terms=on',\n", " '/order?item=drills&name=u~wLUXiowfeCzm{PKyANNDs+{lzNCGfCtQ+RYKuTLxfdTRbl\\\\BMK|x|PTIfQusmpeDlsuo{CV&email=mkjujsfzkjiwfrfhlfesjgfot%40ugtio&city=xw_^EtapK&zip=04571&terms=off',\n", " '/order?item=tshirts&name=_g[MG|eEb|eXioY\\\\vzn{Qqf~Ou_z`^ciVoe}mZsYFRV]~gy+hKDupKRjBuPGfCyNEaheJbuNH}cyOl{O{SJQ`}+MLOuOkdNLYhajrRLExQkxYL\\\\olYPAY{&email=iatfnro%40em&city=aqSuXH&zip=77570&terms=undefined',\n", " '/order?item=tshirts&name=xOmdibaPZimFsSwkFDGmQ+JwzL_tZFn_p^NJDqkN+GzkhZ|b|xu\\\\SxY&email=pncvlnkcxzk%40frj&city=tE}B]JgQl&zip=2451&terms=off',\n", " '/order?item=tshirt&name=zljV`SX[LRgDmk{LreZRmTsIIJ|Dyla^gtQ]LdaDAIHYN+PD~JDMJRmQd{O+Dub}IiaxwcYsHCU{^epIHr{HS~bzLRpa|q{JX]W&email=jvvzkgoxptplcjztai%40usv&city=AI&zip=5248&terms=undefined']\n", "\n", "easy_population_2 = ['/order?item=lockset&name=NvJywJLGfQ\\\\`yfGcJ+JhkHSGO]dO}IuzRR`kYZtwY[Mrim+u\\\\FhGmSNsCzHKLnpbqZzDV[[UDJHd&email=xdhjicxqfbp%40acwuu&city=sc}&zip=7173&terms=on',\n", " '/order?item=drill&name=_}PFXrG]+{uwnfrZCJO~CoKSFAEE+QUGuyi&email=pxblczskyiehfoigrworlrpzh%40dbj&city=yY&zip=8724&terms=undefined',\n", " '/order?item=tshirts&name=AOtv~znUiNso[Zpl_[UtyKihb[eHQ~h`gJcC~BjV+[pKDvDN{][qXwFPUWxZeNXsHvctQiuyHqVPZXW}`oDoidcg+VyTqltmD`iOH~~~kirCPakD&email=vfh%40jz&city=Qqd\\\\&zip=61250&terms=on',\n", " '/order?item=lockSet&name=eZjnTDD[cc|HWgcF_yDCh]OpbfOwExjYbaYrKqrHh~\\\\|x+X_TfD_oMV^NHoziaQoE~wAA+unr|[~Qe~`hIwjgW^Vrx`Ri[pePfIP|TTNhVnC_bmkBY\\\\&email=lhwhyksxjahdhuzzzisovryekvxu%40gfch&city=rtlpUGg_j&zip=62865&terms=checked',\n", " '/order?item=tshirts&name=os^UOAvsXIPqefJhQewKdyFq\\\\MiyVevUB+JF{]\\\\hBw\\\\[E}DBfrpH]aL{ZiC^Hpx[cuEDg~U+mjV\\\\E_vDPWdNw|LuJMuiJDrRnAKVU&email=lznpnqsehsshqivzcodgyakz%40nmpm&city=IxGzQ__CG&zip=04850&terms=checked',\n", " '/order?item=drills&name=rknk_C+mdzLKK~fu[sMBjnK\\\\HNGU`c`ZQFCHAaB{PQ[+LpSQmeOtQ]]fwRvEpi~RbArhplnSsgcd&email=kktyjseabqamrylvmkwughyrer%40ivoi&city=SAVH&zip=50486&terms=checked',\n", " '/order?item=lockseR&name={eXkkRU]{CDg~]\\\\qRpDiQNKpGK\\\\[\\\\F^NE+pMeE\\\\h{rasg_JqZlKx+Luk]FaidweB]iRuP_ynPM~\\\\^JILlM&email=awswefu%40aakn&city=wjcvz`|WH&zip=04237&terms=on',\n", " '/order?item=Tshirt&name=IXXQIn}arq\\\\WfktJhiuxSY\\\\EVOgU{fPq\\\\]NM\\\\+ETBUc`Smo[TEu[nPj[[Y``VttAEdhRQA+bT}z{NTpmVG&email=xpkzsfsmugwa%40ywm&city=jPxn{xg[T&zip=0316&terms=on',\n", " '/order?item=tshirt&name=zNsHbdZKPyJyLmmNLilW+HptpfN^yij[pOyIUfI^hv^lwhBAK+nOsPiGxh`Z`Q{GBeu&email=nyxebhlglhhjvxgrkbt%40ghjiu&city=VGAR_mrl&zip=6203&terms=off',\n", " '/order?item=lockset&name=Lr`rncnb}lT_SqQx\\\\}[xCsEmYkJszWbcxQ_+epxi^{Ux[bjHbYxxfBxkBFcCwGZ+OsJog~fht^|EINPpwc|_~ePTkxW`^lnLoHMJjnPK&email=upqxnsnnntipbljlbaygc%40jm&city=J^GSPx&zip=05325&terms=on']\n", "\n", "easy_population_3 = ['/order?item=tshirts&name=gidT\\\\DZ+sfkiE{^+eLLP\\\\ywMTUN`[peYL]fpu}k]y^N^gT]]F&email=iwlzqtlrophapfwgnk%40dmoqc&city=ky~_rv&zip=88572&terms=off',\n", " '/order?item=Drill&name=lqMl[wKDIGynItJya{^^CekLNxWD_PY}zItmi{_+XQzWdBChYHt_Cjrl_hCoSUARuMeNCJd{KyPm^gcYYCVfYrE+DcAMjIqCt_X&email=ta%40nu&city=E_FVHKIWE&zip=3133&terms=off',\n", " '/order?item=tshirt&name=Tpz^^Y+Hlla}R`dKjnZMW_jR|s+{~SzVoGKBhmCM|Ltnu`ssgaVqetGoWuFIgt~D\\\\V^qZVu&email=tayacvgksixfkglpamagqobleyjfx%40fny&city=Z`&zip=24180&terms=undefined',\n", " '/order?item=drill&name=pkyuDvYjChAF\\\\vPVkt+rPIGiNQdWkZo\\\\zPgXFbjW+Y~CoonnMjDL&email=ynkrozlelleed%40bs&city=yz&zip=84134&terms=undefined',\n", " '/order?item=Drill&name=PAlZWic~s`]uwAVZe+VpURv\\\\+NbjGKATJe]uYyVuSR[SioV|QEjd&email=qsbtjcqwp%40xt&city=eUq_Vwv{u&zip=63666&terms=undefined',\n", " '/order?item=drill&name=VKNPLVq~RvWmMNS+Ip|D}rn]Bd|rrz|qs{xtgD|xHX}tjhOxMdp}{CW{n+|``iZCE}SxvpXDa[cfcHx{o\\\\^yAK~AKX~fgSk&email=fzxuxciqmoqmiefpzh%40xgt&city=CBKRQr{&zip=6228&terms=off',\n", " '/order?item=Drill&name=[_aa\\\\CPIiLw}WmSZLAIOjCXgtPirFzalERFkibqmLN{qc+^\\\\qpF\\\\pDY`SL[wdz_MK}tS]omC]zWzJkldaqjDNn_V`+N\\\\cWV&email=flvsmobmyqqpqclycxq%40gyjki&city=ePVe&zip=6131&terms=checked',\n", " '/order?item=Tshirt&name={FG^[uEnPJZaz_kEN|rc^UiJaPGfuDgHSxpF+YYHbeSjGfCO\\\\}SRN`BLtznRN`ARfyDvYZCz\\\\rvytRc+`i\\\\CwWo&email=hujnvtclucndtojklustszrpe%40jlc&city=ih&zip=4063&terms=on',\n", " '/order?item=tshirt&name=|GL`cs{f}N]uU|VrHiRhh~+bedudSzDuYgDIA~dYTiosmMj_ofyS+QGg{io_zpFfsVXdi_egDEGpLtJzYY&email=blbcswyrelmaffz%40mkmlb&city=RwHJeSH&zip=1342&terms=undefined',\n", " '/order?item=lockseR&name=Nx|rm|\\\\wFx|_+LHy\\\\War\\\\g`+iVoiUD{D&email=caepkhtn%40twcuk&city=Dq&zip=42010&terms=off']\n", "\n", "easy_population_4 = ['/order?item=Tshirt&name=vNUwFBAoyAT+oZ_sm~N_E\\\\V[qeM`TDiGbhS`QgZYhvZVow+zgiR[BJMESzT_Q_W`tOLFoSdP|ggyouSzUGrOnvVaswceS&email=ewgwntvibdrztrdwmuthfsp%40zv&city=nBPxcVf\\\\&zip=20734&terms=on',\n", " '/order?item=tshirts&name=MwljgKvHGWLa|jt~K_+kn^fCwDhfoqcSa\\\\}WsBm|hmkoOwDJF}K\\\\T\\\\h^nU+RVcqfEHeQCMEf}ZGn{gcjxNX|vI|}SU&email=migbpfryy%40yiqzk&city=fl&zip=55703&terms=off',\n", " '/order?item=tshirt&name=M^PPvbjohBmY~lP{aXk]Hi+}efWl_UTFw[rH[Hco_qLW{+K^K}_gf[MEw^iprEyUAj_NnzcE}pr{hgeRzXyJ&email=ayscfyqtmkefnjrxrgr%40qp&city=vvGjZJ&zip=1133&terms=checked',\n", " '/order?item=drills&name=`qPZcXZcy_bPShz`|qoRGez_wkJmaApX~wwzpi[]]+NQ[ANDZA\\\\cBvAfVJ}y^UTXYmo^sf|SoKUPPv]`i+jtvTptCaFbSM`S&email=wuvqyclw%40vtflj&city=]Gs{eG&zip=6628&terms=off',\n", " '/order?item=lockset&name=ZGlUzW}Huwdpsu`JoFhmo~GKic]jpEOG~+BvDVsxZLA^tvJoxphFM+ueToMlNh&email=nfxm%40ot&city=DjCfLp&zip=4132&terms=checked',\n", " '/order?item=drill&name=uD][V+ADSY_bFc[wG^We+]qc`AakXb{aPdc}YjHuKbJ|^ctcfdsq^eOuz^DyBzjo&email=ppbmnnq%40jhdkw&city=TKXy|vJT_&zip=87376&terms=on',\n", " '/order?item=lockSet&name=DOMvOOQVKxLSbQ^AxOXJXrpjUgiym+BWVwgYzAOgsDTIOwS|SPbgQrICM~soqJCeOzdDI\\\\h+SgPZkLp]^q__VNMNI]~uVQqV^z&email=cbvduqcgsyq%40zj&city=Zk}~zA&zip=77175&terms=undefined',\n", " '/order?item=tshirts&name=vqPA_gOwWU`jAHi{JWwD[gX|m}[H+ogrJAbsTAp~~g_ZcqLfZKhyIqdUFfFmzPebwHfAoJ|iZNNM+IKTVzwW^oNq~VOfgpOULUEzobC|&email=hcvb%40vjc&city=Ru&zip=23406&terms=on',\n", " '/order?item=tshirts&name=ojbDYmz]eNe\\\\nAIwllU{F]fZSRM^MExQgZQtTOy_DEJbcSDn+^J]A\\\\^Ry`S^gnWK`trokGKExBBupZtzGNmBPIn_KwC+zx_bz&email=baaq%40dy&city=uJ&zip=0534&terms=undefined',\n", " '/order?item=tshirts&name=]VYBcvLR`ts[iaM{y[R]Z[ZtPnaKufwkSeg+heGEcfP+wtZLWLU&email=zsrzop%40ap&city=XsUrWw&zip=3624&terms=checked']\n", "\n", "easy_population_5 = ['/order?item=tshirts&name=\\\\{`xo+fd{RCW+\\\\zEoMzucwXdSPtkZ&email=gpmxtubhbhygmw%40txzn&city=\\\\]&zip=67647&terms=checked',\n", " '/order?item=tshirt&name=o^^ha}HVwueplVzRbIinGbtqTXTciF|uk]hbd{C{Qobw+ZklST~iqth^RTRuE|PC^\\\\c]\\\\IuJ|r|k_aDJDqE[DUSEF`[BO}+DmzsaCRzNxygMBiHbk]A~B{``eRoO&email=ytczivbt%40ah&city=uYEe&zip=2083&terms=undefined',\n", " '/order?item=lockset&name=z\\\\GSI|KgZ\\\\~_DvTNYgspZO[rmaYyBZ^oeZPcLPWnzBQ`f``k+CXxkkollsRvicqS~z`asIcCzdcET+qDggE~rka`Wzj_JF\\\\QJEFPe~]ZtZPBXrIzliqeW`Jj&email=miuv%40usvlx&city=__ixTp&zip=4640&terms=checked',\n", " '/order?item=Tshirt&name=t]DpVRTManMWVqwGHfs`Hqsd+AuHwHz+nwhtTTQZdfKAHHPfHsFIzSYYjc`hfpQl&email=yzcosirztfimp%40jfd&city=lOne&zip=18244&terms=checked',\n", " '/order?item=tshirts&name=MMbFnySgq|tGLKd~MZhtm\\\\K+~EXR]FPmVDFdMVp}oFOUO{CamzmSSQOXtgrN_E\\\\bP+qh`{VOcbmW\\\\Rya}XY}|oP\\\\x~qSb\\\\^y^&email=ohttbobdghisjhbnaoopd%40oz&city=BnO&zip=2311&terms=on',\n", " '/order?item=drills&name=Gfvao}sPavAfMiIhL{}L+mY~WY_aXJRM]zP}K]ntNaiLrwTO+V``agcopDYJg&email=uxbvyrepotecwegxujcy%40yjd&city=J~q&zip=6622&terms=checked',\n", " '/order?item=lockseR&name=qbWhYW^aD^zGBrmY}PXAo~+Ts`c_T^lRKBT\\\\EaXAdjJBhMLEI+JjeR}dLS[YA]eCCo&email=hvnuga%40dyc&city=XpxbLs}`&zip=6858&terms=undefined',\n", " '/order?item=drills&name=LemXBCgNtv`x|{sDk|Du_GTh+o`NQQG`u^+l{D}IYjjo|sDNmbLM\\\\]ruPRM`u}rGrAODZJsKDx&email=rkehtvf%40iisc&city=cuwoGet\\\\Z&zip=8517&terms=undefined',\n", " '/order?item=Tshirt&name=tqQgwKZ[rungdsR\\\\P~h\\\\|~]`f`Z]_DXVsmUOdKODlzvxr[_+TL`KQjRW{EtuOw|tIcrWWZ`GSRLRxPmQcp`]H+xF_arunUJVpbyod_p]K&email=mfnynrv%40zk&city=WxtBs~&zip=82235&terms=off',\n", " '/order?item=tshirt&name=~HCCsCz`qQ^uMesXDoAP^jlpgKI+g{HW`rofTB+t_aaAVtn\\\\RdVLhuRBjlu^t}\\\\YtqvONdgDXAPCCgKwhRXYK~Es&email=qdceqzurjcwvrbtuy%40mstg&city=GOkCCpdT&zip=17580&terms=checked']\n", "\n", "\n", "\n", "medium_population_1 = ['/order?item=lockseR&name=ldpNi&email=D|YGFOGtogNJXFK`&city=qxwttImFDA&zip=5LaJ1&terms=OfE}',\n", " '/order?item=tshirt&name=E\\\\RZOi{BHhZC~W|`TmD[{^nAADCVqCa~lEAS&email=[leUKirshBP\\\\[btA[~S&city=|BClYiQF_ldAcFJwdxZk&zip=aN98A&terms=OIOE{',\n", " '/order?item=7rill&name=|CLuljs_AHbXqSrrcm&email=u[VJuRreRS~ZZ`nu}ixa&city=&zip=QvpMa&terms=GhWt',\n", " '/order?item=Tshirt&name=POZnyDrtP^qQJIxDUuCBUr`gweG|uxRaYfgGCYkkDN&email=cMVjynq`&city=O&zip=2mHHQ&terms=ppZnY',\n", " '/order?item=Drill&name=RaDh~H`uzXEG\\\\jxBdj&email=DNcUGOGxNKsDJ&city=lvD]pnXRwOi&zip=B0tp4&terms=k[g',\n", " '/order?item=dr5ill&name=giEzDxb[oJ|\\\\TEgQvDoMJoZRvZWFjBxKCu&email=GQwhn|H}MNqqGU}&city=dV&zip=ZX6ts&terms=PwufW',\n", " '/order?item=drills&name=n}`tUgxp~xK[OJWHrHHArtxypa{CerHW\\\\~Bf&email=yZp`qkG^NKpY&city=lNzpyAX&zip=nbGHm&terms=OVRVZ',\n", " '/order?item=dr5ill&name=CFZNHhnY^UlLsBCBUOcub^cRwm[\\\\b]pezvhO~&email=]Nafq&city=FBApNOJ&zip=WUKOn&terms={z~NJ',\n", " '/order?item=dr5ill&name=n]LNlBend}GhwBN~bNAvgd[FZNclFj&email=PZhEybs_xFmWS[`iGL&city=dc&zip=7SriE&terms=goDGd',\n", " '/order?item=lockset&name=CvKuqbzGeIxQ_ZhwQi~qRpQH&email=mrtc\\\\WIsZOvugQWINK[mBsse&city=EiDZfTsBE\\\\tgZcYHRSarQQj&zip=7fK4t&terms=xGIe']\n", "\n", "medium_population_2 = ['/order?item=lockset&name=phJmt^NXl|fzbw]xy[cK&email=KuEueYuq&city=kGJ\\\\QY^SBiD~NmrHkZR&zip=etXBo&terms=}kr',\n", " '/order?item=trills&name=|dZoN|SyWQ{nUnQhoc&email=i}gCqQ|rCCh~&city=fDG[nDhR~XwqJ&zip=3mj82&terms=qh}Fu',\n", " '/order?item=lockSet&name=BNSXtQuNvvm_CfubwPZNjes~DbKDZFWgNT&email=\\\\oTIJFk~BQbZ&city=FAX^tYOPuWxsNKnpTHxsB&zip=m7ufD&terms=~yS',\n", " '/order?item=drills&name=`fyZTMiqRTICENOqIwcxSBHv{eTXFEG_F_Eaz&email=IgeL[zQCJuAr}j[[bK&city=Bj&zip=xlyb3&terms=Bheyq',\n", " '/order?item=lockSet&name=ONyMoRaUK&email=qA&city=QLVsvEdeKc]DjC&zip=jtKPs&terms=MpPt',\n", " '/order?item=lockseR&name=jTs{tkxTxUUKyQ_BY`VIDutbgOmKjtnPPRVoHJyMg&email=L\\\\dhgI{]YGkkMHIganjl}yoaHM&city=x{heToriUuejhqxOP&zip=jLvz8&terms=BQbSf',\n", " '/order?item=trills&name=MJ_|Os~{UiQ^wY`pmQTRh&email=gtmjlU]IFDDfJgQuz^R}YVk&city=j^BRZ&zip=iNMhZ&terms=Nmso',\n", " '/order?item=7rill&name=tAP][QLqYsTWoCM&email=CbioMFmAU&city=PUFBao\\\\e|FUcS&zip=gUj1h&terms=_tC',\n", " '/order?item=PLockset&name=gMDibjCZsSXtpv\\\\rXERmkL`~bsK_Bc^lGzT[&email=^\\\\v`UKIhApPp\\\\TW^}&city=qXDXARUs~C[DLnSpvvmdGbi&zip=Z1w2Q&terms=h_nU',\n", " '/order?item=P0ckSet&name=EMo\\\\NcLZVTcCLha~G]Fwg|dtMNqodrkXWzzYNzD{i|&email=F|~HXUCKmT}hpq^tHSAU^&city=i]gfEj]^h]CcbmdsaO&zip=EOj8n&terms=|A[']\n", "\n", "medium_population_3 = ['/order?item=PLockset&name=mVDDVW{C&email=ZPR`sq|&city=RW~GpprnZaZLM^z&zip=kPhcs&terms=OXON',\n", " '/order?item=drills&name=uSoBG}GOFxSI&email=vcsAf\\\\MFvEcOt`IZdqWXMhfe[D&city=h}_oovisBVG&zip=grPQu&terms=rMh',\n", " '/order?item=Tshirt&name=i{}obHPGEghLdhWeaz]p{DcNT`baSrcxHacEkIjMDSi^d_At&email=ITeXX^u[WMFoXtQtkK\\\\DoDc&city=KPxkKK&zip=N5TvJ&terms=xEeo',\n", " '/order?item=lockseR&name=fdf~Y\\\\SmAvq^hC&email=GEj&city=EQc&zip=WxCJ7&terms=Rii',\n", " '/order?item=lockseR&name=ddZndmKkghOtVtxGJRJlJa|nr^&email=d}kTuPpKt^SU]aCeNs&city=fOpakV&zip=UQjiK&terms=k^knt',\n", " '/order?item=lockSet&name=Atq[x|bn`NPfmHJe|AgOH|mAF&email=dfd|&city=EhoxYD&zip=lZf4S&terms=Q\\\\U',\n", " '/order?item=lickseR&name=jF~aSVaSRUHZKo&email=BiU|E]K&city=g[gvVj&zip=9Y9Sn&terms=XPcjG',\n", " '/order?item=dr5ill&name=\\\\jwvJHLQJA_fub\\\\o&email=g`[JNHBGTa_^LbP}z&city=c}\\\\sqgKO`&zip=KJXPE&terms=|hQTN',\n", " '/order?item=PLockset&name=ikq__nuVd[T|Q&email=DodfHufVEwgnoAKQ&city=e&zip=YU1Wb&terms=xfuww',\n", " '/order?item=lockseR&name=TCUpHm|uPMsIzyuSvkffws|pK^gH_`DIFtM~]Wb&email=H~zBtcj|GCkxkiYF~T&city=le^e&zip=t9Okl&terms=ewxls']\n", "\n", "medium_population_4 = ['/order?item=lockSet&name=QZFGZSySgB~FZeFwV[nYWxuIDlL^jM&email=SOMPtFEUgS{VdeIG\\\\vLYpk}SbJ}m&city=`GAEaLXGRui&zip=EPfoN&terms=\\\\F{Y',\n", " '/order?item=lockSet&name=cSXiF~vcWGvKi|h~EakWCtuM}GhLTWvFpa]wn&email=~aswnHUrNTTQH|Jo|^mOKR&city=kr`P^Dn|W&zip=wdW4O&terms=xfQ_d',\n", " '/order?item=lockset&name=AHdSG^adQJzWrGvdTWd`kOJHkErbW]gEdWmB[qSGZAGD&email=rI&city=zESlhq~xTN&zip=MeBMS&terms=MLVo',\n", " '/order?item=PLockset&name=p__VJf{kiB^[yQFfqfOH]FySTYvExAJgfmAGKDSemC_JcjLW&email=oG}u^NUknIYb[yWj|WlW&city=jpX[atchhCd_FEL&zip=4avtl&terms=xHRAs',\n", " '/order?item=P0ckSet&name=Hrnb|YgzDSYRWNoRsLZ_AKvfmJNMFOEPXLcb&email=vHD]_TkPFF`dnUhQ}ocM[Pj&city=&zip=DRSLb&terms=kpDHy',\n", " '/order?item=lockSet&name=tvtDegQ[J\\\\pPe&email=MbNji~|He&city=vDfoX^CBnhX_n&zip=Ms6GP&terms=Uf`FF',\n", " '/order?item=lockseR&name=ZtSZ}ErZnGB^B]RiFadlukfNEn{aGyRbtPajQxg]W&email=YRWOxcms&city=PnHDgJr{q|m&zip=UzxEX&terms=exzWt',\n", " '/order?item=drills&name=NtTL[YK~f^_P}qwZVip&email=plmK&city=yF}Ig}o~krRUs}[RwOzk&zip=MKY7g&terms=bcOc',\n", " '/order?item=7rill&name=HoLLgQQtngM^mZbmBgUx&email=~VpPBo&city=j&zip=FEOQ6&terms=Cqet',\n", " '/order?item=PLockset&name=WMJcoPILfjuVWJXrHEIQdBwbEO&email=fnSePclqrgDtZmNwlZopnE~K&city=lLwtr_&zip=R0tVR&terms=olvoJ']\n", "\n", "medium_population_5 = ['/order?item=lockseR&name=NUzyZb^lV~v[iXdD`]WcfcVt&email=ZHUr&city=~WlfJK}yiPiWGf^ut&zip=OYyJm&terms=YBzoB',\n", " '/order?item=drill&name=}Yu^]{ZQHEcC`prPveIzstv{gBHUOD^&email=}ZLXRHMDJApxP]{TrdbiXLwk&city=|NC{du\\\\De~}MoPwED&zip=Foz86&terms=WVl',\n", " '/order?item=lockSet&name=QIWt]WvvkxWSkoNXl[T~__Rmjl{enPnoQaKdg\\\\XXrZ^OdF{C&email=R}z\\\\qNWhr|r|{zE\\\\j|h&city=bQVo`XolVHpOCAXRad\\\\&zip=zbvHI&terms=qCAd}',\n", " '/order?item=7rill&name=iQwa}_khox`}yWrtkAwt\\\\AsuYubV&email=yoIBPLSRmpKf]yJmD|G&city=~XbsIDj&zip=jG2gd&terms=Eqk[',\n", " '/order?item=7rill&name=s~LmWN&email=bQ&city=}vYpUIoFHBtl}ay&zip=peBZx&terms=Igzy',\n", " '/order?item=7rill&name=OuV_tLRvCM`elGAku&email=|fxk`LhY^ziaWmniYiInooS&city={PaiJQFh{DLapWBXUib|VY&zip=C3pFM&terms=^EG',\n", " '/order?item=PLockset&name=naCf~[`oAk]JToypvUoHlKWWF&email=Ix|UcwATCXEPrP[UaaRMD`ELrsD&city=W{fZsrkive]&zip=XNrv6&terms=T\\\\n',\n", " '/order?item=lickseR&name=E^UWDSUiMYeLI_hi~]&email=ska^QoeJiDGnJjTo&city=r\\\\U}j|I\\\\MYrQxoH&zip=mEc2g&terms=Sj{',\n", " '/order?item=drills&name=e_Q{hvnT~MxzUgwDGouUDN}aLJ^`ffSwbOiYWXzOLXVxa&email=V]XZFwci`mBkLgnYtMEj&city=Ruz{ZImmg[\\\\~qY~Dqkz&zip=lik9X&terms=SOX',\n", " '/order?item=Tshirt&name=`iZda]ntlUyWKyHZ&email=Ec]qdRA[OyVdWywHq|&city=[`Zj\\\\}Pwn|]CTRHt&zip=9QOVr&terms=D_NV']\n", "\n", "\n", "\n", "\n", "hard_population_1 = ['/order?item=kgEtxX]T&name=奝Ũ㽽\\uf571翫嶼\\udcc8伏ല㒌祄㍨軾䟠黁캎ꋃ&email=剅渽㸾羫ꦌ녿鍯穷烌䌏㷛\\ue12a脓옐\\u17eb\\udad1\\ue46a㌺\\ueab1\\ue484醍ᕮዂ鱡좣\\udb1b煪&city=gGFxPeD}QyJgEUOkB&zip=kTlbJoXG&terms=lSP',\n", " '/order?item=mtTP&name=滧쌧ᥤ櫫啕\\ue1bc쾇甓蒊\\udb8d䣳Ђྨ쩒\\ue372⣼\\udcfcᩧ\\u191f൳ꄄ⺟ꊨ뱸ᄪ&email=⠊ዢꛅ옋ﻘ얎\\uec66픤旱⺨滾\\ueacf컞\\uf666柳ᒕ暴篻⼸弨ﺩ죺ꔏ\\uf877頒嵤ᮘ㩟&city=ofSEfycNuznG&zip=1jZY46gY&terms=LhpiQ',\n", " '/order?item=wIXu&name=焲㩪\\uf4e8♊忣ἢ壜ﯜ냪ﶗ\\uda56ｲ璶\\uf48fṓ꜌ꃻ挰擆ሾ别ᤪ艝\\ue8df橙巐깩Ҹ둾\\ue3ab格뫑\\ue865机沠濔溙\\ueedd倐뿚ﶢ&email=㡤糣&city=BVVeh^gqUmtweUN`KE&zip=N5NlXGT5&terms=PqN',\n", " '/order?item=jgEpuB`Q&name=㳌㍐쎍\\uf12e\\ue6c6\\ufbc7觕遫ષ䖚聶՛㚌㪼ﰫ⎊풐ݫ椛ꑙ묠뾞穗逕歀맩\\ueba9&email=긴\\ue78f샦ꧼ⠷\\udc8c괞陀\\u202dᶢ\\ued5bᏼ㡺\\u0ff3矙ꄈ췞㑀ח톧洿㳑ӿ\\u139c&city=q_&zip=cLoPXNMf&terms=fR^}m',\n", " '/order?item=twotiv\\\\&name=ﾨꝕ寮§悦粠釶略侔돴ꏛ踦ɏ厝莠콁ᡖ皤閕⽷沺ᘂཧ﮾赽灤餖\\uf664ኪ葎꧱굣蟲旦浅疑秲፦\\uf714ย\\ue418꽫\\uf069\\uf16d砰밥&email=\\uf433\\ue55a⸎⟳쉢妑葐\\ua7c2ᵅ莜豺㴕\\uf60b뤍䭻誆䇐&city=uuT\\\\zWBk}ZddtsJ&zip=wJsk2LQr&terms=NHcZ',\n", " '/order?item=Tnp[}_SYv&name=趏榹꿢봢퓩屇쌜\\u1caeႇ凧\\uf621ɯ飓獐ퟔ阥⠍㏣歗&email=ꅀ\\uf0e8\\uea34挥먉\\uf656纝쐰\\ue9e7&city=\\\\Ms&zip=6uMk5gkN&terms=ZXa_',\n", " '/order?item={iAe&name=毙ᐆ咍喸ᡒ&email=\\u0eedਵ궩⳦ꞅ◍═\\u20c4푳믢얛ʸ\\ueb56穝&city=kKw`XOuDYfMyAqAM]&zip=QUcyIxtb&terms=lWjLq',\n", " '/order?item=Oq~&name=崯꼟\\uf561\\uef97鱕\\uf8eaꃏу㵧\\ue238酾&email=鎉\\ue30f\\u0b84ᯄⶨ楏왅̒ᔸ謩迱粋覯ද甄Ӥ쑝窟鈿\\uf405䞷ᗳ짊ᵟ䘆ⷺ&city=bdwNcZmfaafq&zip=RqtUcIz3&terms=AeOY\\\\',\n", " '/order?item=VUy]L\\\\ol]&name=趘\\udf7f䍗\\u0383ᓾ콤轝閞宅坎餤☹ᝥ듕&email=͗ꎴ鱝\\udd14\\u9fe5鏍⅚羛꼀橖首\\udef4\\udd1d\\ueb51㸤\\ua83b詃狾Չ&city=QlPcvQXu_hwshIEmm&zip=BKhN6cdu&terms=l^dd',\n", " '/order?item=qvtzmfR&name=ԃ᳣㤇촹綌&email=䗸\\uda41긍燆⌍뀤㎽鑼舴仟\\uec6e&city=P}wE&zip=TLJN6uV3&terms=fnZ']\n", "\n", "hard_population_2 = ['/order?item=fnCaU]&name=䥊ພ\\uedac澏懺牮\\udad1\\ueb39钞梃튠塉댺毜✞獝✢戓\\uf265䩤㡫\\ud9d8拀㱯ᒐᏑ樚岩鹹ⲧ\\ue0b9翊\\uf357ܗꠧ\\ue88a\\ue412\\uec50\\uf55b&email=\\ue86e\\ue007\\uf367\\ue838ᎌ쳸&city=Xt\\\\rLIrRshk&zip=iEf7zY3z&terms=Bq{ln',\n", " '/order?item=yfr&name=৳엍잓鰔똞⏜蕭鈌끹脏偈穻䱡ᵨ蹫謴\\udc0e舼\\uf5a6ꯡ햳⥄䕲⩿냐⢋韴睙䎫㬱矨䉡ⵙﾸॗ㑰雧潙◨ⰳ췲阢\\uf0ef\\uea93ࢲ돪&email=䑖歜\\udeb1㍕匧娇⤸摮\\ueac6搨獏&city=|JoxX`_FNEPQQF|&zip=OMICjXza&terms=h~bkq',\n", " '/order?item={gnl&name=ﾛ嵘퀻櫖땒姴ꫝኢ휥璇\\uddde聿漁⩑骚ᣬ\\uf179⢷\\ue516\\ue577ᬕ㫥緰欑킺斂沨岚䤵냊\\uefd5ᾣ\\ue743খ༱陋瓴쑟吿ᦋよ冣Ẅ㥭焰硧&email=ᘫ웝\\ue18c祰鉬䅛ไ牉璳︁ﾖꔃ┶螄䚘㛓ӳ༹䧮⡑욦\\uf3aaﯺ㖈얳뀽﹛&city=\\\\MXi~MgtKH\\\\mL[LSz~zZvdE&zip=X1wb6QAK&terms=w{R',\n", " '/order?item=EGIQHVvR&name=ᅱ쉿켧뿱圐&email=ᶓ듻\\uf693祙뜨魽\\uf010쒭놀&city=NQXxMhSxTwJox}^`^h&zip=L7KqcEvB&terms=uDA',\n", " '/order?item=qm_m&name=〶䕼佖蝩슲\\udcf8쬦씸弣ⓥᷴ귲\\u173d憁澸\\ue610뫱콀⓾넃儦겿宒씔&email=\\u0c8d鰯鴞炼&city=DWsJuWEZpifPAclrR&zip=z7Mooc8n&terms=PoPl',\n", " '/order?item=SahNIcnBe&name=\\uf18f⎚ᾦ䜦怖䃘\\ue19d䟏엕茥&email=쒠ￅ藽赧팻鋘脒靊䘐鄷跙骝띡&city={{~FVj^xrdR&zip=4sqU96Rq&terms=BbTVd',\n", " '/order?item=XJ\\\\X&name=焚Ⴔ泏\\u0c5c齺뾼פּ嗲㦊䱋￫䵥樆ç詇ﱇᖎꧧ㫦\\uef67鉰뵙ⵍ梮넭&email=촢蒓&city=kbybtNJVGY^&zip=ODLSSuA2&terms={q[',\n", " '/order?item=D]qO&name=᪁ﰶ酝右鮨嗜\\ue2c8载澆潿᩻猢鼇㔗\\ueebf轇퀛톿ᙥ㭍䝑关㟅谩檤逥&email=\\uef9d\\ud97c＿&city=qUNlQdoCgX_zjjeKig&zip=tgZdzX0g&terms=LCNh',\n", " '/order?item=nXbxr&name=䑖븵澗\\u05ce魍≊\\u0de5\\uf60c蜽諸ⰷᶎ\\u0fe2莲\\uf4a9惌䣆\\ue5b5\\ueac9\\ue224觫ꝅ翉胏&email=坯ẟ缾\\u2bfa䭕㫶똼ࡆ\\ue6b8놇豘징ꔐ뎲\\uefa3縁蔦⃗&city=PPhUSaSrwe}{vicR&zip=MqnwiuJS&terms=wuCo',\n", " '/order?item=e|pX&name=\\u05ceƗ䲶协艼⸂蛍ྲྀᮎ\\uee9a轐롩ᒞ邗\\udfb0\\ue0caᔞ䫞婢笸\\udd20覈衍禯\\uede6ꁋ躗Ս쩺躮ﺢ潩绅ּ甌쩊&email=旍\\uecbe肛湁ģ칾ᰈᴵ遽뾾霛깄ᇹ踸혔끣囘Չ分鼬斮Ϫ&city=qY[ahztvk&zip=oRAlKBlF&terms=`mPd']\n", "\n", "hard_population_3 = ['/order?item=hIGC&name=﮳\\u0382犸竖穖%ᬌ탱枖力鸦ꉰ掑穃䵬枡䮞\\u171d됮&email=ꡊع\\ue215䵖짱괒&city=E~uoB\\\\nGftJCk_ch|aAU&zip=tHx7HOmR&terms=gPdCm',\n", " '/order?item=mL|z`RlI&name=嘷삨\\ueb15㽵岳蝠벧ᗫ䪭ଈ髫\\u0fe2䯯矱쳯稖퐝꺥\\udd96엛㱕ꠀ潡\\uee41軼ᬳ柛ᇕঋ\\udf9c㌏㗡㶄쵁줡\\ud93b\\ueeeb&email=벵ᑨ⁐㟪͘䊹❀ⰿ簎ၡ陊䕾䁆䒐頷٩념&city=mQJyv}G~s_EPY&zip=aJr4aEKi&terms=ItQQv',\n", " '/order?item=fUv&name=㠐憦䵺엂\\u0c4f㰫㉶\\ue8fa㲻\\uf481\\uf8ab㒱⼀뒳\\udc58✊嵉찁됲䗪鲵짯\\ue565룙࠽綇깚▉\\uf080\\ud8c0\\ud8ff닦ꈬ&email=ⷃ賞鈴컀⋫棣蒝餻㾌鵿貙桥ẓ\\uef0f﨩霰&city=sX~d|YlnYQ&zip=B4seZEoF&terms=CLUkB',\n", " '/order?item=eO~xIo&name=爫凴潙蘰\\udc66ሆՊ攋ɤ\\ue4fb蓒ᕻᜅ멭ꅏ版甠䟱还挅뿆잂\\ufa6f\\ue1d8鎛\\ue27d죳髑哽숅쇵짞\\udbe8뒏雒Β⩜䆿ꚉ괠탺ᷦ葞趲ʦ럠&email=ﶢ踅㨷瓭枍屷\\ue0f4绚⣌ஶ䋈\\ud80d源诔䔶譹쓹둮똣綑㭰⎾&city=i~l]YKdEkNsZbnMYuNfTO&zip=ELEfisYm&terms={QYvu',\n", " '/order?item=Wrx^nYved&name=뷤䡶ⴾﺠ\\ud95dꩀⰓ\\ue040⾾䘅\\ue490邲錇爍픀㠸列Ĵ\\uf4f1髞৪㴴䴌縨뱰昚แⳔ鏡\\uf2d8畗&email=᥀秕\\ud8f3蜻&city=xdQGCHOgilXv[JI&zip=pT7K6XYm&terms=ZCt}E',\n", " '/order?item=Ja_Od&name=⛻筚ᏀɁ弞ﵜ\\uef2f皀㧢嬛ﶷ칈\\uf275鵍ﴮᓬ殈脎휅\\uf5bd玪磺&email=恦䕩\\uef85ꦫ\\u1979頕몇ଳ뗷튮轀\\udcd9瞦㍲썀ᤡ깐彗&city=ZGwDGZxsyGwj|qgme`bkHY&zip=Hw7LwPZZ&terms=CQRq',\n", " '/order?item=T\\\\Vn&name=尌縗暹⠣վ\\uf5c2⧻큛璺㋉瘁셀䞂ᜂ嬝᱕誠&email=민鳙ᷣ练ₔ☑㉋ம䀧ⶸ\\uf5c8刑ؿ䂺⾥솁酆ꭳ✿㡲\\uf4f9\\ueaa4&city={FIKiMEeq&zip=hkjrR6fq&terms=tum',\n", " '/order?item=~qAUx^O}q&name=\\uf1d9\\uf00c틗ሠ菴\\uefd2\\ue902腙䔞Ϩﳰ\\uf089ਙ퓚Ƅ勇\\ue859\\ue69bᗜᗉฅﹲ뙰녈˜톮॓輵ᗫ대注逶\\uec56\\u245f퐟鹶揢\\udabe쩚怍黷&email=帡ៜ\\ue11b둌&city=nDS\\\\Bb|Xoh&zip=jMQ6kMNe&terms=zt]EL',\n", " '/order?item=^IEwv\\\\D&name=톤㊤\\uf2e9壬挽鱠缾跥ꯌ砺醆ꔸ㈳囤诧ᦣᡵ亞ꘌ❣㧛ῷ祻钘韩쌵绐\\uea3c\\uf6bc\\uec3b멨偣颿낶쎎敳偋갪&email=췱ퟘ聱膷極\\udd1aퟷ璎\\uf3feឋ⼧&city=unAPJZ~MGD_{&zip=wB5uoWDJ&terms=ETpc',\n", " '/order?item=U\\\\|]^C&name=Ƶ\\ue4c0\\ue599ᰢ왇爛◚\\ue0c2כּ衐沚㛼\\ude38㾨䦹䵃ȇ隕ꐵ&email=ꐂ韺䊆劘兮斓霠愡ɰ倄촸៦貘䄟\\udb49\\uf065䳝悫쉵&city=oeUdHmeeAPNR{UnQ^&zip=qNoy9kHG&terms=Ikh']\n", "\n", "hard_population_4 = ['/order?item=ufr&name=浊\\ue7a6反ꄞ蜾ᴹ翿\\udd73犄䟟諕㮱츫ᱩ鰤嗓哯괈ᎀ庾怇謄䶯畬\\uf79a\\uda03턼苊뇞ঽ┟餆&email=\\u169dᤓ쀐蹝棉崧䷣귻ሄ縬솮\\uf352헣妋∞ꐱ쟒ㆉ銋\\udaad&city=ZrgbXZ&zip=Qdl4YAQE&terms=rvJG',\n", " '/order?item=qLPR}BG&name=円尀ⅾ译兠覭&email=잎⨱⭙⅀歄婋㸼ऑ嫎&city=jyI]_&zip=A5oyGmQO&terms=fey',\n", " '/order?item=eLkRkZ&name=ﮠ読\\ufbcd\\uf098\\u1ae1Դ\\uf3ef쌚ᯨＤ黅襤胒탃牛&email=〦\\u2cf8翤辝勞浝䪵發๔㛒᜔å\\udc55\\uddd1춥룔摠\\u0e86羱롶쩱쳮\\ueb4aỵ鳯ꔪ並ޤ&city=wY_gjw&zip=sGQzukZl&terms=hwLv',\n", " '/order?item=LA]WVLm&name=嫧䑤屑븤읨ẕ叉&email=☊Ủᢼ䫎倭얲㰯Ꮠ㎈\\uec90\\ufe67꺘髒꣫&city=Uu\\\\^~\\\\\\\\baH_{P{ey`e&zip=oxTJGr2n&terms=HiuKC',\n", " '/order?item=^aB[&name=ﾑ䋞豌稟䘔왲\\uece5㒎㳩弨翞\\u1aeb좢\\uf034ꭡ筞㵪\\ue36e鮶髛㭿諀糖髂蘔쿰Ϛ\\ue501祉虽䴒礡ᬹᳱ솇뽓&email=㶄\\uf760⻓愈ￖ㨡\\uec02\\ude24強蜃퇍&city=eq_g\\\\S&zip=1CXtlNfa&terms=ntPl',\n", " '/order?item=yuiU&name=徽鱖\\uf558䎷뻩ᘏ䋂ኯ⚍췽䚌᧾䧤\\uee1d䭓鵊咫\\uf540㽃㡐榼ᕗ蓈⇇䖞鮹\\udb77쪛&email=̳聊뀧싍뎅鱅倃\\ud859嚪䚴ᡴ&city=JVWe__c&zip=3UtCAe8e&terms=F`VY',\n", " '/order?item=y^rFn&name=⥅摥♋\\ud851過꧋籴范\\uebdb\\ua7c9苫쵢쭝छ밎槤〨접䣈伶迉ⱕ㵅펓铈팲郞\\u0bc4ﴲ鹙ꀬ媳&email=팊ﺲ疮㐂擒ᝰˍㅀ夆菋橝뽯\\uf49d餤痩掞\\udce0—&city=rbgErYkP}w~CZwq\\\\TYasN&zip=1oKMRVfE&terms=lka',\n", " '/order?item=i|CyRgZ&name=㝢䃏䴏怏햺蔳ⴇ똯乢ꇄ腎魕㔚㭈珀莻威\\uef63䣦뱄ꢿಬ&email=\\uea35㊷ꖢ\\udfc3\\ue676&city=IyG\\\\{Vb&zip=VdLbVH3j&terms=Qdqu',\n", " '/order?item=Olg|[Yx&name=뛁彂괒֟憌ᆂ詜쥃濷㏀︎䥱漰氭쌇࿁柼觵载ᴦ갮ꂊ䒼ᑷ\\uf54c鉈鯂惖䂧袱萸\\uf518ꆡ웊\\uf86d⟵\\uf4cdಐ╻泲鈼陭띇&email=﨑韴狯\\u20c3ඖ繴ă儸뀨㳉唳誌투㟕华㭸楖왚紴њ&city=IwkogMzWuAqoVCvUraD~Jb`&zip=d4Ol0E9S&terms=wtVC',\n", " '/order?item=ANXJ{jAK&name=曯\\ue43fڄ촅掽ड़휟ᖦ\\u085d剢쯦㭧갩\\uec59仗듌惠\\uea6a馜&email=뾉תּ苁谍굡茁⻈㸯美⺩ᢴ삄&city=Suog^&zip=L1yt7Dfo&terms=rA}Se']\n", "\n", "hard_population_5 = ['/order?item=xQjaSnMM&name=ꆡ⢊훿선ឲ椉圬瘛昙䇆ῩƄ昊몡ꅕ癸磅&email=蒏\\ue3e8艆榖ﾔࡈ\\u0b0d&city=}Xbzv[WizSW~u&zip=1Eojkeem&terms=CM{ZQ',\n", " '/order?item=ywLI&name=\\ue997鶻ꆂ㴊ꓝ⋷⺧㙮壖ꈑ䅸爢柏ꀕ죗ꑇ穎&email=\\ue50f苡ݸ爬樯鏪ᙅ&city=[{wq^zfGUdSyqcJBl&zip=PoKd9ENE&terms=qXj',\n", " '/order?item=va~PeCT}&name=ٵﶗ㺤ﱨ﹉㩎椵ꪉ俬铡缦놶촟\\ufdcf먉쟬跰蛷⩻慰磉暇䖾쮺፞珤\\ue92a溛彦ꔉ&email=\\ued90ࢱ㹧ປ砹䳎\\uee11&city=}HPocs_LcnS|M`bn&zip=63ZcAweU&terms=L_]G',\n", " '/order?item=xewB^spR]&name=廙\\udb2bꔔ伔ẹ\\uf4d6⏐웫\\u0e68ᐣ수\\udc49䘊铉潐秇윽一\\u1ae4쑺ळꙜ㤸炭\\ue6cb煭銋톒犍䀞거울\\uee28ꖆᅢ䁒&email=豊ḳ缢紒䢕\\uea3eා〟⠽䯥檈ꗄ\\uf009䨼䓢ㅃ㰗Ꝏ๗ꐯ㨃舎\\ude21&city=Oti_vld`^tl{FHIE^tIWR&zip=Qoi2P3Zu&terms=xvEoc',\n", " '/order?item=EQM]Yaa&name=\\udee6ॠ椔䟠б\\u0efd쓔쯀䭴ᐄ\\ue7faỮ\\uecdc\\u05fe텻┥ⰹﰝ滮燡軼躔ᆧ\\uf4ed䏽䨋鬭&email=\\ue15d萡牞較뛌߁‘殎벨굢㉩샮憒秓ྖ턼ᄎੳ䅁㶽踗曍뗯쒠꾷\\udd53魕鎼㯼&city=aZjo^dkNkmUdlxCrLxWOYRvk&zip=4eU323Po&terms=fzp[',\n", " '/order?item=dMPXYKiTb&name=뻐➁鋱㠐띄퀠쎮&email=\\ud919鸍\\uf2f5⳻뀳ꥣ\\u0bda卶\\uf224೬\\ue6e5쩞椿퍭렴&city=t{|cqJN}~FuaiOP{s&zip=keFuprwW&terms=R^VQ',\n", " '/order?item=gJ|RBmK|]&name=瓠簇鱅䂃㊱䆸⇯\\u193f\\u0eec魷ꛕꬒᷞ\\ua879퉍㩨攁滧跏믖&email=\\x1e糧췲&city=Eu\\\\cudkUww&zip=O1KGcaYc&terms=V^Je',\n", " '/order?item=fMHaMVsiT&name=詤\\uda9bᮎ\\udd0a僅쁊쁡\\ue0c7䩶쉿᷌띤⇐Ⴙ沀ﯫ험篗&email=ೆ郗㣴㳿ፇ\\udf25∓ᱍ줬&city=CwkYxzeIVH|RL&zip=7tRSF6PE&terms=\\\\Y~',\n", " '/order?item=OrUpP&name=蠍檒\\ue1f7′\\ueb65駭䀆\\u1f7f篹춣䍪\\ud8c5濞㧹圷ꞗ㲡䀫陴멢\\uf79c吘쭘\\u2beaﳡ椲\\u2451쀠际ｾ櫶ҡ\\u2bfc헸\\ue0ea䅑\\udc80鱈옷&email=힁狪訪&city=TG{E}Y`YSasSuYVJa&zip=DMuDN2tD&terms=Ytu',\n", " '/order?item=FGDth&name=ꮎ럣鵤襕䞛\\ue938⍿\\udc0b\\uf6f0镰酃屪뿗㋀㮮樤镫묛\\u1ad3汌瓔憶㟧\\uab10◈뒝ᑀ薈爘ʆﭨᖾﻒ뒰&email=䗴楉鯵嬥漂\\ue456\\uded5邘\\uf5a9\\uedcf\\ue861企滊䱦ў㚹씄徑\\u0ee5\\ua4cc푗嵍ꌃ署덍銺큳潶\\ua63f&city=cccQupDCHS&zip=fjQMEpHz&terms=n^RO']\n", "\n", "complex_population_1 = ['/order?item=\\uf87b\\ue310\\udbc6㊋姻䏙朶&name=\\udc52翇㹧㣂\\ueff1А\\ud8cb\\u2001됕䃍꼧\\uf478䭚펂쑐ㄒᙰ禳द愩&email=\\u2e66흽썯\\uddf6&city=揎&zip=*xNoUh)P\\\\h&terms=ﴚ슡韼쳜',\n", " '/order?item=㑩ﱚ⢃&name=髪ᕓꎛ沠達卖氾ੱ낛샒呞䬉\\u09fd堳鳍祡渎\\ue956津ᶙ쌆\\uf4ed碫磀\\uf20bຶ&email=␕텙\\ue112\\udb86⌵옼ꯒᢽ긂楩\\ue542當\\ue8ff倪酃&city=磫剾ር\\uf8f4瑑䄿锭냝賲\\ua7ba绽\\ue697䚒\\udb34潦飝폁&zip=p*#s2{ZKo&terms=䡴',\n", " '/order?item=&name=\\ueee2昍襁䑜\\uf063櫗庴ꉱꝒ욒免ᖼູ妾ྊх琢ᖺ垬粺뤈繀\\uf41b㱊\\uf57a갫&email=湂姽&city=恽&zip=HoebJmpqCh&terms=ᗁ']\n", "\n", "complex_population_3 = ['/order?item=熹&name=\\x0e굝Ᵹ贈⮽妽蓉귲츍镄莸龿\\ue1e0쯌ᇡ䍎铲\\x84ᒬ\\ud96e༪鰊蓙\\u31e6ꈉ\\u0e7b\\ue869婒岐뽶梦০彜\\u0e68ጉጎ襼鰻䆵抇踕뚟ⷣ&email=偍\\x05줚\\udb58\\uf6c3洫\\uebd9끔\\uebf0첀▂㌗\\uf022㍛ᰏ䷒硓햌Ňᘱ葺ᙚ&city=㡧蚷\\udbf7ꡕ쏓\\ue156엠ꦂ\\u0adcΓ⌻ꚦ\\uf315\\ue6b6싛ןڒ㯒&zip=%q87u?{S6Y&terms=뎝빅⯄㫯',\n", " '/order?item=&name=퐝\\udd9a\\u2e5e⇮交夷\\udbe4᪳훏丐㚩ꖇ╿ᑦ論&email=腩ꊽ몝䷰갩✶跂ᷬ蓳訧坺ẃ킖炫\\ue443睫\\ue3ab꾵艰睝&city=\\ue451䶑⬮敭覍⚬뙐✺ì왌薆\\ue532Ⱞ\\u0dff甩ཽ倖\\udd9c\\ue595쵄⣮谱&zip=ou2N82x]v1&terms=撵✾',\n", " '/order?item=\\u1ccaꢣ뷭퐔쐾卸脱\\udcc1㘕&name=\\ue737旷춭乾⫪繌\\ue423녢\\udd77滺녦ꍋ뛁냡쒫喕ꞰⲟŲ枡퇫\\ufddb侩\\uea28眊ᨉ䋆筜噒烨腀邬&email=쑔&city=\\uef76諐瀌䟪냄\\uecb1鬐&zip=t/|\\\\=)Cc?a&terms=㔎䣪蝽鏐',\n", " '/order?item=&name=䎼眩穹䷨ꝩ㌥贵\\udc04펦㴬䧤斂橕戻姀䀵萃좏覌諡윫ࠫ\\ue0fb뮷\\uea33沼賳갫榲䶗碸覒ꉋᒦིﴲ▒蓁荦\\ue2dc&email=\\ueb0f똩㰒\\udf2bㆂ譛\\uf61e\\ufff0Ⱉ椲ᤤ䔳⪯ⶖ騿䔂ࠃ睒&city=\\uffff&zip=8~K91!E9PS&terms=\\ue651㶐摹욊\\ue1b8',\n", " '/order?item=칯萁ꖬ\\uf515䖰縛̎燙&name=\\uaaff棢ǣ䮩\\uf036煟ⶈ㍳Ỳ裘躀곌\\ued40딙ꭆ䅃༉\\u243a㽁䄑褣胖\\uf5c1㕒\\uf1f0픤م麗\\ud92d⼊跮ᥢ恎冈频橑㪪粠睦뤸&email=\\ueec7ங翷씨轲鹐䛒憇逸\\uec17&city=咴ीۍ횚\\u0e7d\\udf2f泼\\ueae9팽箂᪨䷶⋟䐳냽褳&zip=gVyw:CXO<&&terms=姴',\n", " '/order?item=㰱&name=䩼굉먡ꌤ낿됨ꧼ䰲踷គ霨鋻\\ua63bﻆ\\uf8f0ⶌ듴쇥鑲ƒ䚚쿁ン썶抍㰃䰀ූ蘑홚쨰\\ud95d홫鏏\\uf1d2䭍ꔐ&email=硹늕귴A皖腰좚䋧\\ue3da&city=륵蝓ﰪщ&zip=3\\'P)o#,HX\"&terms=ͺ\\ue1f4',\n", " '/order?item=䕇蕀\\ueffc擄&name=\\uf689\\udc62솝\\uf088&email=\\ue514섻釁垰튜磙ꓖ枏&city=䱶\\u0893ౙ䕙\\udaa8胶ྩ\\ued13돝貓뀩꼼㾼Ѳ凑皏ʉ慥穚裃䤋⦏릹&zip=!#^&FhvO1l&terms=',\n", " '/order?item=㻉惍㚛칤痏촊&name=ᒌ凼\\uee7b★䋭&email=ꢒ髫섥㕮䄧儡ྻ詀囅ﴝ䰑ᅘ꦳\\ue877큿隚⛁蜢嘓큖Ꭰ&city=\\uec1c헟圻켞뻫ᷱ⢧洊䮖ᔑ⻍&zip=w_a,#*6P=T&terms=ꍘ艅岇⏴㼹',\n", " '/order?item=殪ᵽ뾋뇚⒕\\ue1a8鶈\\uf079&name=鋗넜噚&email=贤\\udf70븩袢콠ꨓ甂驂ࡋ\\u0c77䍍鶩䷧쉳&city=莧⤇忦䀮拜⧗ꭖ\\ue246ᷫ컳\\u20cb决떄&zip=koIX>r<*E$&terms=',\n", " '/order?item=\\u0ef8ܙ㶸\\u0b04❼뫻맀佈&name=\\ue9aa蠤\\ue910骴ဤ髨蒓爽埜력梟睑숊쯁㷹ᮝ횙✒烪쮁쥆鳌鶜ꋁ㛓閼螢\\ud879곓\\uf3e4䝍\\uf397롞\\ue2a0&email=蚾䋮⳿\\uea30&city=㡾㪜\\ue596\\udb0e릞ߺ礪&zip=r~:yfiKW94&terms=ᯙ\\uec0dꈔ汫㘏']\n", "\n", "complex_population_4 = ['/order?item=价㎝&name=怂嚨䒩\\ueb18셔럊ꑛ\\ufefe婑ﶿÿƨ&email=낕ꃠﱔᳳ⥀ᬸ&city=䕍\\ue577礈焧ff㫾엱\\uef13&zip=BjV4zD8ERU&terms=ᔨ媪',\n", " '/order?item=웤\\uf7c3ᄼ\\ue574롊㉵&name=檹\\uf23f两콝遢꺒䧴ؖ\\ue2ec쌀륺靵蚅諸敁２앴幑឵펾䬳孮冮勍ம퉋ᆝ搜왐䜧鈕똬䫉꼂븚촲\\uecc0\\ue284砶䂻겐ా챔짨맊ﲣ&email=錹ች娮㋥⣓㱾㮖\\uf079譽觑枕꺏\\udb12憮&city=绠抻虛힃ᙡι긻잪ᶠ빼멗ౕ␏ᬰ縪&zip=gEd73^NGQC&terms=퀌',\n", " '/order?item=\\uf45c喭&name=ྲƧ犚\\uf6e1븼ﲤ趸喠᭹晔穒坵跨楣瘇\\u09d0饘\\udb3a带稇\\uf848䗏佈\\uedb6蓠嶿ﰴ䯻ʰ鶅ὧ\\ue436䚻ㆬ喉F舎嬔裲\\uf826&email=톆\\ua7e4朠븄决偻ᾛ峁鬝ȶ㳝&city=쯮꽉⸋苯⊅镯ť诐Ԟ篢葼&zip=!S*g%Mlh>D&terms=\\ue718㌫晜랶',\n", " '/order?item=㉱ҵ\\ue03d掹&name=\\uf6c0䩕٧㊉￨鼶黢愙৸ম莿᷃닲୪⃞਼݈忢澐㬸䗴\\ua6ffᴭ♤\\udc15魼耏\\ud810٥&email=\\u2b74⭸䰓嫥큝ﭹ롃솪⏲呥\\uebc1硋셝琐鸜\\ue1a5Ⰰꞩ铜鬒\\uef4a&city=쑹&zip=K}\\\\7!j#*bH&terms=㑭陕삖伣',\n", " \"/order?item=뉫\\ued79&name=鼣劭\\ud8c1䳢\\udf4fힰ凇Ϻ᳢㦬\\u1cb3釴綯ㇺ\\ue52c્ɵ࠴鳀ⓞ鹯\\udb13轁욓嘮\\ue24f綍ꉟ叒㉯\\uf84a鲈\\uef8bታ⠾滧र&email=쯭ﮨ槊Ⳙ⢏샟꾍㍔匴菟䒣\\uebfb뀔놸\\ue6c2馼䉓椙␠䕂㯎\\uf6f1쐸鋯抅炯㿷晜&city=煌㜓&zip=Q4'@uDu/{V&terms=윇ꀿ鈾\",\n", " '/order?item=殖鳈ꑑ폳ပ&name=ἂ﮹單懍ﯽ锼\\ue316\\ue263\\uf80f뾽蠲ꍿ࡙ᑐ耯뙸\\udb53䙜\\ude08眫펀驸䁾宀㤳誀馷ⵛ釘壁ꗐ誼끓禬&email=⣈\\uec5d冏꿅桝ᔸ睘뢂픾泱᧾&city=ຫ༯Ꮸ᯳ꇲוֹ쵓෧蓝疩囀吾&zip=fXtUWl_}Ou&terms=险쇤ᩒ',\n", " '/order?item=&name=ᾌɐ↊쪎ṣ\\u0fe0\\ue294饯芮鵳Ҁ西╟\\uee19ꬱ膞Վ瞮\\udaf0먣䠱芐\\udc3dꨪ\\uf016폫緾䟜㞰邵럽ꏲ邹ᠺ齍⒰形&email=㺕曣\\ueaed霳\\u0a63뜊妭\\uf4ba悷㗬⺜䘘梇㺫엟\\u0cd0嘥믽&city=轚ꄹ䡫&zip=o,{i2J1]7D&terms=',\n", " '/order?item=\\uf502ࠁｐ\\ue50c&name=乵垈읍飀튍끁ਝⵖ䝚\\ue3ff䬴쒚뼠⎘︧\\ue77c⯁濘\\uaac4쪌&email=ა뗩槁\\x83ᮩ宥彭䤅\\uf25c\\udf2c騌\\ue334쪽듟䦇ﶻ㉭젖疥ᦶ鬟\\ueca0䃖㻔ꍖ&city=텵鹟耶ʷ読䧧&zip=@WB(C2oIRS&terms=㨤㲺㹰⛛',\n", " '/order?item=Ն\\ue736놴烁눬崵䑌&name=褒\\uf5b9髰\\ued41뙅\\ueb14䄘칒㑳\\udd3f\\uf340ᰮ㐩꽀ﴣ⅗\\uf5c3绹&email=㹙똾㧢㇝㪐\\uebdcⅸ春ꦡ\\ua7c1댇햨᳟殨죥罸夐ꆄ&city=\\ud99d⪚耓\\ueec1Ὂꋸ\\uefb8&zip=$dr,D*H3&0&terms=镋쿋픺쾖觐',\n", " '/order?item=와瓞&name=\\u0e7a絮湏\\uee90搄\\uf104ᎋ\\ue1a3鱅ﮡ꼚赱蝍✲䥱✾쒿䈩㖲㨺㞊㯵瞸鯁辛沨Ō灬\\uaafc狑ী\\ue57f隮蘳Ꙇꃟ自猽\\ueda6劬\\ue2e6㹨&email=䴜괞&city=慒ဧ幧ᱝ錏ল㍈芓䡙㱛\\udadc♪䎣\\ue72b弊\\u2bdd归笡⏫&zip=dSP_Zq_[lp&terms=蜵败赶']\n", "\n", "complex_population_5 = ['/order?item=퐕\\ue503&name=Ⴄ蜕\\udd73ꝩ季\\uea38黝\\u0c5c賞굺\\ue1f2\\udfd9쥦燧Šޜ\\uefbc\\ue8b4꙽앚ⱥ왇\\ud898ᲄ⽸驈㓞\\udc5d䖋밊⮰餎Ṧ⨁촱্\\uec5f&email=㶻蚑癝쬐&city=⫇黨雏\\udda4ቀ宵\\u0af6䇒ᥙ뗫\\ue620줺喰섚㽜䠧铮쇂\\ueeb5셌罕߉ﯥ⠀&zip=Mp~:UG%&,`ckV)w&terms=蓒',\n", " \"/order?item=\\u1fdc읻솠\\ue149巀欀ɹﱫ&name=碑态刷潩堒果䳎騥䲎쥻啜첁虔ऺ윸ᚶ몜䥺慮ᱫ敕兝鞖ꅙ몸\\ufb45㹿뭶\\ue806몚됇淙洃蠼\\ue2b5琴濶\\u1acc쨄&email=⟺ꤓ๛ʞ뵫槡踤먑ई㮺\\u2439\\ue15d⻣듞믮쫙\\u1cba孢溚&city=ཪ뉋❜\\ued18䃯&zip=9>'Nls_@:L&terms=\",\n", " '/order?item=&name=丶簩⪳궯獵듥\\u1a8a\\ud889ટ偤\\uec4a䟱꼷彊觙糌옿ජ鑑鋘䲘⭁&email=즍ꑂ蕵쥑㼼詔㆞\\uf5fe␒최ᕢ猎̞&city=柵첋\\ufb0d뎞꜕ꋿ&zip=co\\\\wJKsTMQ&terms=\\udfb7쉏ᣁᙇ㎓',\n", " '/order?item=\\ue1c8&name=巠ꔪ⦵ឿ課㜢\\uef86삐麡鈚殯쩭Ⱈ碣䥷氕꤉뿦ൌ調\\ue60a⥎蚭ㇺㄥ\\ue604ᾄ딆軗츏ซ葸\\ua97d큼䎸삽&email=炢筊Ż㘖ᒠ令䏖ᤲ쳝먣㬂&city=藶\\udbc9䨝遶邺컂ꛐ멈鯉륔뺦䩈쑻⓭驅钮껠棇Ἵ긶쾓졥윢懓&zip=6Er]e.f=sV&terms=诃쪻',\n", " '/order?item=䢮ꯜ蚦\\udae2䜼\\uee53齃\\uf17b&name=譨쎤\\u07fb舫ƽ㹴略斴遑콖栱챘毙悺᠈餽䁘䧕睵醯⥖\\ued68臅梁譧〯徿ꎇ戫湐\\u0ea0౺侢ꨎ曗惆羰\\uf44f᯾碟&email=ᆨ쮕Ɽ\\uede9\\ue5ee섗\\ue689&city=㜘칍茒\\ufbcd抠현\\ue28cꜳ&zip=Ac6tTEB1aj&terms=掛텔᷉',\n", " '/order?item=&name=춏较던懾瘧Ⲣ\\ue06f越✏\\ue2f5짭\\ue392\\u0ef7榋漢ሮ፟\\uf3c2ꩌٳ\\ua7f6㭑퓩앑\\uec26욷专ᷕ뙔фᚤ\\ued12驫確槺ᦧ웍ឡ涖\\uead9䜒2暹㿐龖垅肙ᝍ&email=糜\\uedce\\ue89d\\uf6ecꚾ䦗脫譮䫢⨱Ꮑ沯䟭\\uef37粎␕ⳟ䬝痙\\uea19誸✝號鑱貔&city=丣ః빝흝䧵䫹뺗笹잃ꃓ㻬熜&zip=([WXZ9#B%h&terms=哞뽩',\n", " '/order?item=另\\ue13e꧖\\u173f&name=\\uf435牲\\ue883ᣕ䝈ꄡ\\u1bf8直멞Ꝅm敐\\ue489魨东Ă俹\\udf38ᡀ뙄\\uaafeヾ五푞糴\\udec4ᾨ欘穆庴꓂ⓖ불&email=ꐹ\\uf1b3玉&city=㔲⥃䡷&zip=Grp?3aO=JV&terms=넋욈眐츻',\n", " '/order?item=笐샨&name=ᛀ鴍螫磶瞚➬여䶟즮ꐟ樳⍲祊\\u171aꦊ訄\\uf658\\ue8f4맅ﺑ&email=軳균쐺蹭촱⚸锭ꗽ&city=鳿籂᩶똹추䁰棻\\ue10c\\u0ba7＼割盕脀䠍䃱ㆶ郂ⷨ폆橅ﮊꦞꉧ&zip=Vwf4E[b?d]&terms=塥覎審鵢',\n", " '/order?item=\\udbaf&name=㈼\\uf25a\\u181b常ө폸\\udf46㰡鸹ᮭ⨡唛燸捣\\ue313䣺&email=볨\\u0cdd覣檍홨猩⟼쑦Ⰵ撀⚌木\\uf636䕕驝듉\\uf7df紻虰籠\\ud949&city=틵푁惝︧쥲\\udc74퐉ꝶ¢❺❠헾\\ufffaఽ럧霙\\ue92c妹瘫ꎧ&zip=-*3WV?&p2X&terms=䩌周']" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [], "source": [ "initial_population = [easy_population_1, easy_population_2, easy_population_3, easy_population_4, easy_population_5, \\\n", " medium_population_1, medium_population_2, medium_population_3, medium_population_4, medium_population_5] \n", "\n", "harder_population = [hard_population_1, hard_population_2, hard_population_3, hard_population_4, hard_population_5, \\\n", " complex_population_1, complex_population_2, complex_population_3, complex_population_4, complex_population_5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Scoring\n", "\n", "For each URL input in the population and its corresponding target, your implementaton should generate a list of test inputs, __maximum of 10 inputs__. These inputs will be executed on the server and graded based on:\n", "\n", "* Number of iterations of your GA algorithm (less is better)\n", "* Number of reached target pages , i.e. error pages, confirmation page and page not found\n", "* Number of resulting valid inputs for each target page.\n", "\n", "\n", "__Note__\n", "* __For each target, we expect at least _one_ valid generated input to pass the target, out of maximum 10 generated inputs__.\n", " \n", "* __Your implementation will be tested with a _secret_ (possibly more complicated) population which may contain inputs other than the ones provided in the project.__ " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Grading framework\n", "\n", "For grading:\n", " * __To pass__, your implementation should produce valid inputs that reach the confirmation page, page not found and all error pages for the `easy_population` and `medium_population`.\n", " __Points__ will be assigned based on the number of reached pages, reaching the confirmation page, number of resulting valid inputs and the number of iterations of your GA algorithms. \n", " * __To obtain extra points__, your implementation should produce valid inputs that reach the confirmation page, page not found and all error pages for all or some of the `hard_population`. \n", " __Points__ will be assigned based on the number of reached pages, reaching the confirmation page, number of resulting valid inputs and the number of iterations of your GA algorithms.\n", " * __Bonus Points__, your implementation should produce valid inputs that reach the confirmation page, page not found and all error pages for all or some of the `complex_population`. \n", " __Points__ will be assigned based on the number of reached pages, reaching the confirmation page, number of resulting valid inputs and the number of iterations of your GA algorithms." ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [], "source": [ "def compute_results(population):\n", " resulting_population = [] \n", " num_generations = 0\n", " num_valid, num_invalid, validity_score, reachability_score = 0,0,0,0\n", " \n", " reachability_list, validity_list = [],[] \n", " gen_list, pass_list = [], []\n", " \n", " dash = '-' * 115\n", " print(dash)\n", " print('{:<8s}{:<8s}{:<10s}{:>8s}{:>12s}{:>12s}{:>18s}{:>12s}{:>20s}'.format(\"pop_id\", \"reg_id\", \"target\", \"#gens\", \"#val\", \"#inval\", \"reachability\", \"validity(%)\", \"Pass(1)/Fail(0)\"))\n", " print(dash)\n", " \n", " itn = 0\n", " num_regex = 1\n", " \n", " for reg_ind in range(0,num_regex):\n", " \n", " list_regex = [itemList, name_regex[reg_ind], email_regex[reg_ind], city_regex[reg_ind], zip_regex[reg_ind], terms_regex]\n", " httpd_process, httpd_url = start_httpd(ProjectHTTPRequestHandler)\n", " \n", " for test_population in population:\n", " itn += 1\n", " for target in target_list:\n", "\n", " try:\n", " resulting_population, num_generations = genetic_algorithm(test_population, target, list_regex) \n", " except Exception:\n", " pass\n", "\n", " if len(resulting_population) <= 10:\n", " reachability_score, num_valid = get_web_reachability(resulting_population, target)\n", " num_invalid = len(test_population) - num_valid\n", " validity_score = (num_valid/len(test_population)) * 100 \n", "\n", " gen_list.append(num_generations)\n", "\n", " reachability_list.append(reachability_score)\n", " validity_list.append(validity_score)\n", "\n", " pass_list.append(bool(validity_score > 1))\n", "\n", " print('{:<8d}{:<8d}{:<10s}{:>8d}{:>10d}{:>12d}{:>16d}{:>16d}{:>16b}'\\\n", " .format(itn, reg_ind + 1, target, num_generations, num_valid, num_invalid, int(reachability_score), int(validity_score), \\\n", " validity_score > 1))\n", " else:\n", " raise ValueError(\"Number of generated inputs ({0}) is greater than 10 \".format(len(resulting_population)))\n", "\n", " total = len(population) * len(target_list) * num_regex\n", " \n", " per_passed = (sum(pass_list)/total) * 100\n", " per_failed = ((total - sum(pass_list))/ total) * 100\n", " reachability = sum(reachability_list)\n", " avg_validity = (sum(validity_list)/ len(validity_list))\n", " avg_gens = sum(gen_list)/len(gen_list)\n", " \n", " return per_passed, per_failed, reachability, avg_validity, avg_gens" ] }, { "cell_type": "code", "execution_count": 51, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-------------------------------------------------------------------------------------------------------------------\n", "pop_id reg_id target #gens #val #inval reachability validity(%) Pass(1)/Fail(0)\n", "-------------------------------------------------------------------------------------------------------------------\n", "1 1 ISE_INC 55 10 0 20 100 1\n", "1 1 ISE_Item 0 8 2 16 80 1\n", "1 1 ISE_Name 0 2 8 4 20 1\n", "1 1 ISE_Email 17 1 9 2 10 1\n", "1 1 ISE_City 28 1 9 2 10 1\n", "1 1 ISE_Zip 17 7 3 14 70 1\n", "1 1 ISE_Terms 35 10 0 20 100 1\n", "1 1 CONF 30 1 9 5 10 1\n", "1 1 PNF 64 10 0 10 100 1\n", "2 1 ISE_INC 98 10 0 20 100 1\n", "2 1 ISE_Item 0 6 4 12 60 1\n", "2 1 ISE_Name 0 4 6 8 40 1\n", "2 1 ISE_Email 26 1 9 2 10 1\n", "2 1 ISE_City 41 1 9 2 10 1\n", "2 1 ISE_Zip 43 1 9 2 10 1\n", "2 1 ISE_Terms 126 1 9 2 10 1\n", "2 1 CONF 206 1 9 5 10 1\n", "2 1 PNF 108 10 0 10 100 1\n", "3 1 ISE_INC 74 10 0 20 100 1\n", "3 1 ISE_Item 0 6 4 12 60 1\n", "3 1 ISE_Name 0 4 6 8 40 1\n", "3 1 ISE_Email 19 1 9 2 10 1\n", "3 1 ISE_City 20 1 9 2 10 1\n", "3 1 ISE_Zip 63 9 1 18 90 1\n", "3 1 ISE_Terms 49 1 9 2 10 1\n", "3 1 CONF 142 1 9 5 10 1\n", "3 1 PNF 53 10 0 10 100 1\n", "4 1 ISE_INC 50 10 0 20 100 1\n", "4 1 ISE_Item 0 7 3 14 70 1\n", "4 1 ISE_Name 0 3 7 6 30 1\n", "4 1 ISE_Email 19 1 9 2 10 1\n", "4 1 ISE_City 26 4 6 8 40 1\n", "4 1 ISE_Zip 110 1 9 2 10 1\n", "4 1 ISE_Terms 50 1 9 2 10 1\n", "4 1 CONF 79 1 9 5 10 1\n", "4 1 PNF 140 10 0 10 100 1\n", "5 1 ISE_INC 48 10 0 20 100 1\n", "5 1 ISE_Item 0 7 3 14 70 1\n", "5 1 ISE_Name 0 3 7 6 30 1\n", "5 1 ISE_Email 38 1 9 2 10 1\n", "5 1 ISE_City 77 3 7 6 30 1\n", "5 1 ISE_Zip 69 7 3 14 70 1\n", "5 1 ISE_Terms 41 1 9 2 10 1\n", "5 1 CONF 59 1 9 5 10 1\n", "5 1 PNF 87 10 0 10 100 1\n", "6 1 ISE_INC 144 10 0 20 100 1\n", "6 1 ISE_Item 0 8 2 16 80 1\n", "6 1 ISE_Name 0 2 8 4 20 1\n", "6 1 ISE_Email 56 1 9 2 10 1\n", "6 1 ISE_City 42 2 8 4 20 1\n", "6 1 ISE_Zip 44 1 9 2 10 1\n", "6 1 ISE_Terms 50 1 9 2 10 1\n", "6 1 CONF 82 1 9 5 10 1\n", "6 1 PNF 115 10 0 10 100 1\n", "7 1 ISE_INC 138 10 0 20 100 1\n", "7 1 ISE_Item 0 9 1 18 90 1\n", "7 1 ISE_Name 0 1 9 2 10 1\n", "7 1 ISE_Email 6 1 9 2 10 1\n", "7 1 ISE_City 53 1 9 2 10 1\n", "7 1 ISE_Zip 82 1 9 2 10 1\n", "7 1 ISE_Terms 46 1 9 2 10 1\n", "7 1 CONF 86 1 9 5 10 1\n", "7 1 PNF 57 10 0 10 100 1\n", "8 1 ISE_INC 147 10 0 20 100 1\n", "8 1 ISE_Item 0 10 0 20 100 1\n", "8 1 ISE_Name 1 1 9 2 10 1\n", "8 1 ISE_Email 717 1 9 2 10 1\n", "8 1 ISE_City 43 5 5 10 50 1\n", "8 1 ISE_Zip 253 1 9 2 10 1\n", "8 1 ISE_Terms 368 1 9 2 10 1\n", "8 1 CONF 260 1 9 5 10 1\n", "8 1 PNF 385 10 0 10 100 1\n", "9 1 ISE_INC 121 10 0 20 100 1\n", "9 1 ISE_Item 0 9 1 18 90 1\n", "9 1 ISE_Name 0 1 9 2 10 1\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "9 1 ISE_Email 12 1 9 2 10 1\n", "9 1 ISE_City 44 1 9 2 10 1\n", "9 1 ISE_Zip 21 1 9 2 10 1\n", "9 1 ISE_Terms 43 1 9 2 10 1\n", "9 1 CONF 59 1 9 5 10 1\n", "9 1 PNF 107 10 0 10 100 1\n", "10 1 ISE_INC 131 10 0 20 100 1\n", "10 1 ISE_Item 0 9 1 18 90 1\n", "10 1 ISE_Name 0 1 9 2 10 1\n", "10 1 ISE_Email 31 1 9 2 10 1\n", "10 1 ISE_City 24 1 9 2 10 1\n", "10 1 ISE_Zip 66 1 9 2 10 1\n", "10 1 ISE_Terms 37 1 9 2 10 1\n", "10 1 CONF 95 1 9 5 10 1\n", "10 1 PNF 151 10 0 10 100 1\n" ] } ], "source": [ "per_passed, per_failed, reachability, avg_validity, avg_gens = compute_results(initial_population)" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Percent passed: 100.00%\n", "Percent failed: 0.00%\n" ] } ], "source": [ "print(\"Percent passed: {0:.2f}%\".format(per_passed))\n", "print(\"Percent failed: {0:.2f}%\".format(per_failed))" ] }, { "cell_type": "code", "execution_count": 53, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "avg % validity: 43.33%\n", "avg gen_list: 70.27\n" ] } ], "source": [ "print(\"avg % validity: {0:.2f}%\".format(avg_validity))\n", "print(\"avg gen_list: {0:.2f}\".format(avg_gens))" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Reachability: 710.00\n" ] } ], "source": [ "print(\"Reachability: {0:.2f}\".format(reachability))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.6" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }