{ "cells": [ { "cell_type": "markdown", "id": "6c06000b-f238-478b-ab60-672e511358bd", "metadata": {}, "source": [ "# CPE" ] }, { "cell_type": "markdown", "id": "f86bac1a-41aa-470f-8d6a-99d89703de66", "metadata": {}, "source": [ "**Common Platform Enumeration (CPE)** is a structured naming scheme for information technology systems, software, and packages. Based upon the generic syntax for Uniform Resource Identifiers (URI), CPE includes a formal name format, a method for checking names against a system, and a description format for binding text and tests to a name. This method of naming is known as a well-formed CPE name (WFN)\n", "\n", "> source: [cpe.mitre.org/specification](https://cpe.mitre.org/specification/)\n", "\n" ] }, { "cell_type": "markdown", "id": "e905d329-eb7e-4552-94ce-1c8985f6e966", "metadata": {}, "source": [ "You can see this notebook directly via:\n", "- [GitHub](https://github.com/LimberDuck/limberduck.org/blob/master/docs/notebooks/cpe/cpe.ipynb)\n", "- [Jupter nbviewer](https://nbviewer.org/github/LimberDuck/limberduck.org/blob/master/docs/notebooks/cpe/cpe.ipynb)" ] }, { "cell_type": "markdown", "id": "d18ea856-b88a-4144-a015-c621bf1f208a", "metadata": { "tags": [] }, "source": [ "## Generation time" ] }, { "cell_type": "code", "execution_count": 1, "id": "0b6493e1-214a-475c-aff7-84e76f802de7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2024-04-16 05:54:00 +0000\n" ] } ], "source": [ "from datetime import datetime, timezone, timedelta\n", "\n", "timezone_offset = 0.0\n", "tzinfo = timezone(timedelta(hours=timezone_offset))\n", "generation_time = datetime.now(tzinfo).strftime('%Y-%m-%d %H:%M:%S %z')\n", "print(generation_time)" ] }, { "cell_type": "markdown", "id": "e4a65b9b-d993-495b-92f4-463d544f9ca1", "metadata": {}, "source": [ "## Creative Commons" ] }, { "cell_type": "markdown", "id": "ae373a12-06be-41be-bea8-c01904517c0b", "metadata": {}, "source": [ "This notebook and generated diagrams are released with [Creative Commons liecense (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/deed.en).\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 2, "id": "f660068b-6834-49b7-9dd5-ed149a4ed2d2", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cc.xlarge.png\n", "by.xlarge.png\n" ] } ], "source": [ "import requests\n", "import urllib3\n", "\n", "urllib3.disable_warnings()\n", "\n", "urls = ['https://mirrors.creativecommons.org/presskit/icons/cc.xlarge.png',\n", " 'https://mirrors.creativecommons.org/presskit/icons/by.xlarge.png']\n", "for url in urls:\n", " file_name = url.split(\"/\")[-1:][0]\n", " print(file_name)\n", "\n", " file = requests.get(url, verify=False)\n", " open(file_name, 'wb').write(file.content)" ] }, { "cell_type": "markdown", "id": "7af4b2f4-d6ed-4327-89ed-f0080ff9e88c", "metadata": {}, "source": [ "## CPE data downloading\n", "\n", "All CPE stats are taken from [nvd.nist.gov/products/cpe/statistics](https://nvd.nist.gov/products/cpe/statistics)" ] }, { "cell_type": "code", "execution_count": 3, "id": "d46bff47-b63d-4d2d-b634-21cac2b2c254", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "16\n" ] } ], "source": [ "from urllib.request import urlopen\n", "import ssl\n", "from bs4 import BeautifulSoup, SoupStrainer\n", "\n", "def get_data(url):\n", "\n", " ctx = ssl.create_default_context()\n", " ctx.check_hostname = False\n", " ctx.verify_mode = ssl.CERT_NONE\n", "\n", " page = urlopen(url, context=ctx)\n", " html = page.read().decode(\"utf-8\")\n", " \n", " product = SoupStrainer('table')\n", " soup = BeautifulSoup(html, \"html.parser\", parse_only=product)\n", " \n", " return soup\n", "\n", "url = \"https://nvd.nist.gov/products/cpe/statistics\"\n", "data = get_data(url)\n", "\n", "print(len(data))" ] }, { "cell_type": "markdown", "id": "9974ad6b-3ee7-483c-9226-370d241bd3ac", "metadata": {}, "source": [ "## New CPE entries" ] }, { "cell_type": "markdown", "id": "44f06c78-39e5-40cf-a298-16f7c84bcbc6", "metadata": {}, "source": [ "### CPE data parsing" ] }, { "cell_type": "code", "execution_count": 4, "id": "bae10707-ccbc-4683-9090-5da17daf9506", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " | Year | \n", "Summary | \n", "January | \n", "February | \n", "March | \n", "April | \n", "May | \n", "June | \n", "July | \n", "August | \n", "September | \n", "October | \n", "November | \n", "December | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | \n", "2009 | \n", "5786 | \n", "181 | \n", "422 | \n", "830 | \n", "639 | \n", "612 | \n", "209 | \n", "241 | \n", "110 | \n", "330 | \n", "102 | \n", "1960 | \n", "150 | \n", "
2 | \n", "2010 | \n", "11094 | \n", "990 | \n", "234 | \n", "509 | \n", "1106 | \n", "803 | \n", "886 | \n", "426 | \n", "1252 | \n", "707 | \n", "1336 | \n", "1026 | \n", "1819 | \n", "
3 | \n", "2011 | \n", "9263 | \n", "702 | \n", "925 | \n", "583 | \n", "2252 | \n", "818 | \n", "682 | \n", "430 | \n", "390 | \n", "423 | \n", "470 | \n", "737 | \n", "851 | \n", "
4 | \n", "2012 | \n", "23235 | \n", "1148 | \n", "1003 | \n", "1622 | \n", "2501 | \n", "3228 | \n", "2259 | \n", "3124 | \n", "2887 | \n", "1583 | \n", "1830 | \n", "1110 | \n", "940 | \n", "
5 | \n", "2013 | \n", "15671 | \n", "1460 | \n", "1105 | \n", "1138 | \n", "1271 | \n", "932 | \n", "989 | \n", "1623 | \n", "1359 | \n", "1140 | \n", "1941 | \n", "1327 | \n", "1386 | \n", "
6 | \n", "2014 | \n", "16714 | \n", "1552 | \n", "1151 | \n", "1644 | \n", "2103 | \n", "2175 | \n", "1447 | \n", "1606 | \n", "886 | \n", "885 | \n", "260 | \n", "1380 | \n", "1625 | \n", "
7 | \n", "2015 | \n", "6503 | \n", "758 | \n", "654 | \n", "849 | \n", "838 | \n", "429 | \n", "736 | \n", "446 | \n", "564 | \n", "81 | \n", "216 | \n", "144 | \n", "788 | \n", "
8 | \n", "2016 | \n", "11074 | \n", "2450 | \n", "535 | \n", "871 | \n", "699 | \n", "732 | \n", "1471 | \n", "321 | \n", "762 | \n", "979 | \n", "452 | \n", "696 | \n", "1106 | \n", "
9 | \n", "2017 | \n", "18650 | \n", "746 | \n", "888 | \n", "1070 | \n", "1935 | \n", "1494 | \n", "1210 | \n", "1364 | \n", "1889 | \n", "1516 | \n", "1596 | \n", "2585 | \n", "2357 | \n", "
10 | \n", "2018 | \n", "56509 | \n", "2273 | \n", "2404 | \n", "1571 | \n", "2525 | \n", "10489 | \n", "5160 | \n", "3707 | \n", "3895 | \n", "3722 | \n", "10862 | \n", "6564 | \n", "3337 | \n", "
11 | \n", "2019 | \n", "272322 | \n", "6810 | \n", "7197 | \n", "10466 | \n", "12979 | \n", "13181 | \n", "32837 | \n", "45459 | \n", "23763 | \n", "47195 | \n", "26717 | \n", "22731 | \n", "22987 | \n", "
12 | \n", "2020 | \n", "171681 | \n", "15960 | \n", "18668 | \n", "20159 | \n", "16306 | \n", "14157 | \n", "11250 | \n", "10659 | \n", "14459 | \n", "13077 | \n", "10921 | \n", "9293 | \n", "16772 | \n", "
13 | \n", "2021 | \n", "189565 | \n", "11253 | \n", "14969 | \n", "18562 | \n", "15476 | \n", "13997 | \n", "18730 | \n", "20791 | \n", "16531 | \n", "14022 | \n", "16164 | \n", "17160 | \n", "11910 | \n", "
14 | \n", "2022 | \n", "175793 | \n", "11299 | \n", "12125 | \n", "13566 | \n", "10410 | \n", "10262 | \n", "14432 | \n", "13652 | \n", "18251 | \n", "14397 | \n", "15560 | \n", "14888 | \n", "26951 | \n", "
15 | \n", "2023 | \n", "223030 | \n", "18297 | \n", "16483 | \n", "22071 | \n", "14262 | \n", "17591 | \n", "19858 | \n", "18929 | \n", "20943 | \n", "16827 | \n", "21344 | \n", "18296 | \n", "18129 | \n", "
16 | \n", "2024 | \n", "39737 | \n", "20260 | \n", "16220 | \n", "2058 | \n", "1199 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "