{
"cells": [
{
"cell_type": "markdown",
"id": "6c06000b-f238-478b-ab60-672e511358bd",
"metadata": {},
"source": [
"# CPE"
]
},
{
"cell_type": "markdown",
"id": "f86bac1a-41aa-470f-8d6a-99d89703de66",
"metadata": {},
"source": [
"**Common Platform Enumeration (CPE)** is a structured naming scheme for information technology systems, software, and packages. Based upon the generic syntax for Uniform Resource Identifiers (URI), CPE includes a formal name format, a method for checking names against a system, and a description format for binding text and tests to a name. This method of naming is known as a well-formed CPE name (WFN)\n",
"\n",
"> source: [cpe.mitre.org/specification](https://cpe.mitre.org/specification/)\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "e905d329-eb7e-4552-94ce-1c8985f6e966",
"metadata": {},
"source": [
"You can see this notebook directly via:\n",
"- [GitHub](https://github.com/LimberDuck/limberduck.org/blob/master/docs/notebooks/cpe/cpe.ipynb)\n",
"- [Jupter nbviewer](https://nbviewer.org/github/LimberDuck/limberduck.org/blob/master/docs/notebooks/cpe/cpe.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "d18ea856-b88a-4144-a015-c621bf1f208a",
"metadata": {
"tags": []
},
"source": [
"## Generation time"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "0b6493e1-214a-475c-aff7-84e76f802de7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2023-03-30 19:34:20 +0000\n"
]
}
],
"source": [
"from datetime import datetime, timezone, timedelta\n",
"\n",
"timezone_offset = 0.0\n",
"tzinfo = timezone(timedelta(hours=timezone_offset))\n",
"generation_time = datetime.now(tzinfo).strftime('%Y-%m-%d %H:%M:%S %z')\n",
"print(generation_time)"
]
},
{
"cell_type": "markdown",
"id": "e4a65b9b-d993-495b-92f4-463d544f9ca1",
"metadata": {},
"source": [
"## Creative Commons"
]
},
{
"cell_type": "markdown",
"id": "ae373a12-06be-41be-bea8-c01904517c0b",
"metadata": {},
"source": [
"This notebook and generated diagrams are released with [Creative Commons liecense (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/deed.en).\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f660068b-6834-49b7-9dd5-ed149a4ed2d2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"cc.xlarge.png\n",
"by.xlarge.png\n"
]
}
],
"source": [
"import requests\n",
"import urllib3\n",
"\n",
"urllib3.disable_warnings()\n",
"\n",
"urls = ['https://mirrors.creativecommons.org/presskit/icons/cc.xlarge.png',\n",
" 'https://mirrors.creativecommons.org/presskit/icons/by.xlarge.png']\n",
"for url in urls:\n",
" file_name = url.split(\"/\")[-1:][0]\n",
" print(file_name)\n",
"\n",
" file = requests.get(url, verify=False)\n",
" open(file_name, 'wb').write(file.content)"
]
},
{
"cell_type": "markdown",
"id": "7af4b2f4-d6ed-4327-89ed-f0080ff9e88c",
"metadata": {},
"source": [
"## CPE data downloading\n",
"\n",
"All CPE stats are taken from [nvd.nist.gov/products/cpe/statistics](https://nvd.nist.gov/products/cpe/statistics)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "d46bff47-b63d-4d2d-b634-21cac2b2c254",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"15\n"
]
}
],
"source": [
"from urllib.request import urlopen\n",
"import ssl\n",
"from bs4 import BeautifulSoup, SoupStrainer\n",
"\n",
"def get_data(url):\n",
"\n",
" ctx = ssl.create_default_context()\n",
" ctx.check_hostname = False\n",
" ctx.verify_mode = ssl.CERT_NONE\n",
"\n",
" page = urlopen(url, context=ctx)\n",
" html = page.read().decode(\"utf-8\")\n",
" \n",
" product = SoupStrainer('table')\n",
" soup = BeautifulSoup(html, \"html.parser\", parse_only=product)\n",
" \n",
" return soup\n",
"\n",
"url = \"https://nvd.nist.gov/products/cpe/statistics\"\n",
"data = get_data(url)\n",
"\n",
"print(len(data))"
]
},
{
"cell_type": "markdown",
"id": "9974ad6b-3ee7-483c-9226-370d241bd3ac",
"metadata": {},
"source": [
"## New CPE entries"
]
},
{
"cell_type": "markdown",
"id": "44f06c78-39e5-40cf-a298-16f7c84bcbc6",
"metadata": {},
"source": [
"### CPE data parsing"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "bae10707-ccbc-4683-9090-5da17daf9506",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"
\n", " | Year | \n", "Summary | \n", "January | \n", "February | \n", "March | \n", "April | \n", "May | \n", "June | \n", "July | \n", "August | \n", "September | \n", "October | \n", "November | \n", "December | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | \n", "2009 | \n", "5786 | \n", "181 | \n", "422 | \n", "830 | \n", "639 | \n", "612 | \n", "209 | \n", "241 | \n", "110 | \n", "330 | \n", "102 | \n", "1960 | \n", "150 | \n", "
2 | \n", "2010 | \n", "11094 | \n", "990 | \n", "234 | \n", "509 | \n", "1106 | \n", "803 | \n", "886 | \n", "426 | \n", "1252 | \n", "707 | \n", "1336 | \n", "1026 | \n", "1819 | \n", "
3 | \n", "2011 | \n", "9263 | \n", "702 | \n", "925 | \n", "583 | \n", "2252 | \n", "818 | \n", "682 | \n", "430 | \n", "390 | \n", "423 | \n", "470 | \n", "737 | \n", "851 | \n", "
4 | \n", "2012 | \n", "23235 | \n", "1148 | \n", "1003 | \n", "1622 | \n", "2501 | \n", "3228 | \n", "2259 | \n", "3124 | \n", "2887 | \n", "1583 | \n", "1830 | \n", "1110 | \n", "940 | \n", "
5 | \n", "2013 | \n", "15671 | \n", "1460 | \n", "1105 | \n", "1138 | \n", "1271 | \n", "932 | \n", "989 | \n", "1623 | \n", "1359 | \n", "1140 | \n", "1941 | \n", "1327 | \n", "1386 | \n", "
6 | \n", "2014 | \n", "16714 | \n", "1552 | \n", "1151 | \n", "1644 | \n", "2103 | \n", "2175 | \n", "1447 | \n", "1606 | \n", "886 | \n", "885 | \n", "260 | \n", "1380 | \n", "1625 | \n", "
7 | \n", "2015 | \n", "6503 | \n", "758 | \n", "654 | \n", "849 | \n", "838 | \n", "429 | \n", "736 | \n", "446 | \n", "564 | \n", "81 | \n", "216 | \n", "144 | \n", "788 | \n", "
8 | \n", "2016 | \n", "11075 | \n", "2450 | \n", "535 | \n", "871 | \n", "699 | \n", "732 | \n", "1471 | \n", "321 | \n", "762 | \n", "979 | \n", "453 | \n", "696 | \n", "1106 | \n", "
9 | \n", "2017 | \n", "18658 | \n", "746 | \n", "888 | \n", "1070 | \n", "1935 | \n", "1498 | \n", "1211 | \n", "1364 | \n", "1890 | \n", "1516 | \n", "1596 | \n", "2587 | \n", "2357 | \n", "
10 | \n", "2018 | \n", "56522 | \n", "2276 | \n", "2405 | \n", "1573 | \n", "2525 | \n", "10491 | \n", "5163 | \n", "3707 | \n", "3896 | \n", "3723 | \n", "10862 | \n", "6564 | \n", "3337 | \n", "
11 | \n", "2019 | \n", "272333 | \n", "6810 | \n", "7197 | \n", "10466 | \n", "12979 | \n", "13181 | \n", "32838 | \n", "45459 | \n", "23766 | \n", "47202 | \n", "26717 | \n", "22731 | \n", "22987 | \n", "
12 | \n", "2020 | \n", "172042 | \n", "15960 | \n", "18668 | \n", "20159 | \n", "16307 | \n", "14517 | \n", "11250 | \n", "10659 | \n", "14459 | \n", "13077 | \n", "10921 | \n", "9293 | \n", "16772 | \n", "
13 | \n", "2021 | \n", "189646 | \n", "11253 | \n", "14969 | \n", "18562 | \n", "15476 | \n", "13997 | \n", "18730 | \n", "20795 | \n", "16536 | \n", "14038 | \n", "16172 | \n", "17167 | \n", "11951 | \n", "
14 | \n", "2022 | \n", "176239 | \n", "11317 | \n", "12151 | \n", "13618 | \n", "10428 | \n", "10272 | \n", "14470 | \n", "13817 | \n", "18272 | \n", "14418 | \n", "15577 | \n", "14904 | \n", "26995 | \n", "
15 | \n", "2023 | \n", "56957 | \n", "18322 | \n", "16517 | \n", "22118 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "