{
"cells": [
{
"cell_type": "markdown",
"id": "604150b3-255c-441f-8f67-c95b2f91158e",
"metadata": {},
"source": [
"# CVE"
]
},
{
"cell_type": "markdown",
"id": "9688bdc8-ceb9-4fb5-a43b-588a5f108d83",
"metadata": {},
"source": [
"**Common Vulnerabilities and Exposures Identifier (CVE ID)** is a unique, alphanumeric identifier assigned by the CVE Program. Each identifier references a specific vulnerability. A CVE ID enables automation and multiple parties to discuss, share, and correlate information about a specific vulnerability, knowing they are referring to the same thing\n",
"\n",
"> source: [www.cve.org](https://www.cve.org/ResourcesSupport/Glossary?activeTerm=glossaryCVEID)"
]
},
{
"cell_type": "markdown",
"id": "bb012dcd-8476-4501-8ca2-1008a08588e3",
"metadata": {},
"source": [
"You can see this notebook directly via:\n",
"- [GitHub](https://github.com/LimberDuck/limberduck.org/blob/master/docs/notebooks/cve/cve.ipynb)\n",
"- [Jupter nbviewer](https://nbviewer.org/github/LimberDuck/limberduck.org/blob/master/docs/notebooks/cve/cve.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "f7c29080-90bd-4e34-bc6f-92511ed31595",
"metadata": {},
"source": [
"## Generation time"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "ff06696a-18c2-4c59-9cae-bc0dd8b7b308",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2024-04-16 05:31:46 +0000\n"
]
}
],
"source": [
"from datetime import datetime, timezone, timedelta\n",
"\n",
"timezone_offset = 0.0\n",
"tzinfo = timezone(timedelta(hours=timezone_offset))\n",
"generation_time = datetime.now(tzinfo).strftime('%Y-%m-%d %H:%M:%S %z')\n",
"print(generation_time)"
]
},
{
"cell_type": "markdown",
"id": "f3a4c46a-1ece-4601-9f72-90d64e12f888",
"metadata": {},
"source": [
"## Creative Commons"
]
},
{
"cell_type": "markdown",
"id": "33983601-bf85-4ba0-babc-5e3a69bc5ef4",
"metadata": {},
"source": [
"This notebook and generated diagrams are released with [Creative Commons liecense (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/deed.en).\n",
"\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "17811d3a-f62b-4c35-9bad-75fcdc9e9cf5",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"cc.xlarge.png\n",
"by.xlarge.png\n"
]
}
],
"source": [
"import requests\n",
"import urllib3\n",
"\n",
"urllib3.disable_warnings()\n",
"\n",
"urls = ['https://mirrors.creativecommons.org/presskit/icons/cc.xlarge.png',\n",
" 'https://mirrors.creativecommons.org/presskit/icons/by.xlarge.png']\n",
"for url in urls:\n",
" file_name = url.split(\"/\")[-1:][0]\n",
" print(file_name)\n",
"\n",
" file = requests.get(url, verify=False)\n",
" open(file_name, 'wb').write(file.content)"
]
},
{
"cell_type": "markdown",
"id": "f0d55e25-0b30-4377-95cf-20f471fcaf21",
"metadata": {},
"source": [
"## CVE data downloading"
]
},
{
"cell_type": "markdown",
"id": "b29a1112-344a-4015-91d1-c1ee0aa63629",
"metadata": {},
"source": [
"All CVE IDs are taken from [cve.mitre.org/data/downloads/index.html](https://cve.mitre.org/data/downloads/index.html)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "239ee776-f15c-43d4-a8f5-d1ca251f0f37",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"allitems.xml.Z\n"
]
}
],
"source": [
"url = 'https://cve.mitre.org/data/downloads/allitems.xml.Z'\n",
"file_name = url.split(\"/\")[-1:][0]\n",
"print(file_name)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "50f24f67-75e2-4d85-bf23-47182cbf46f3",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"69886215"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import requests\n",
"import urllib3\n",
"\n",
"urllib3.disable_warnings()\n",
"\n",
"file = requests.get(url, verify=False)\n",
"open(file_name, 'wb').write(file.content)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "e02dc4ee",
"metadata": {},
"outputs": [],
"source": [
"import unlzw3\n",
"from pathlib import Path\n",
"\n",
"uncompressed_data = unlzw3.unlzw(Path(file_name))\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "8b59663f",
"metadata": {},
"outputs": [],
"source": [
"with open(file_name[:-2], 'wb') as file:\n",
" file.write(uncompressed_data)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "d2ff78a8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"allitems.xml\n"
]
}
],
"source": [
"import glob\n",
"\n",
"file_name = glob.glob('*.xml')[-1]\n",
"print(file_name)"
]
},
{
"cell_type": "markdown",
"id": "cb262f32-6398-44c9-a365-d5e1b47dfcd8",
"metadata": {},
"source": [
"## CVE data parsing"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "69608e9f-cbad-40db-85f2-48d25d1aa381",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" number year\n",
"0 CVE-1999-0001 1999\n",
"1 CVE-1999-0002 1999\n",
"2 CVE-1999-0003 1999\n",
"3 CVE-1999-0004 1999\n",
"4 CVE-1999-0005 1999\n",
"... ... ...\n",
"311256 CVE-2024-30266 2024\n",
"311257 CVE-2024-30267 2024\n",
"311258 CVE-2024-30268 2024\n",
"311259 CVE-2024-30269 2024\n",
"311260 CVE-2024-30270 2024\n",
"\n",
"[311261 rows x 2 columns]\n"
]
}
],
"source": [
"import pandas as pd \n",
"import xml.etree.ElementTree as et \n",
"\n",
"tree = et.parse(file_name)\n",
"root = tree.getroot()\n",
"df_cols = [\"number\", \"year\"]\n",
"rows = []\n",
"\n",
"for item in root:\n",
" item_name = item.attrib.get(\"name\")\n",
" item_year = item_name[4:8]\n",
" rows.append({\"number\": item_name, \"year\": item_year})\n",
"\n",
"df = pd.DataFrame(rows, columns = df_cols)\n",
"\n",
"print(df)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "ecbe6644-37e1-4747-b8cc-3181570bfb1e",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"
| \n", " | year | \n", "number | \n", "
|---|---|---|
| 1 | \n", "1999 | \n", "1579 | \n", "
| 2 | \n", "2000 | \n", "1243 | \n", "
| 3 | \n", "2001 | \n", "1573 | \n", "
| 4 | \n", "2002 | \n", "2436 | \n", "
| 5 | \n", "2003 | \n", "1603 | \n", "
| 6 | \n", "2004 | \n", "2779 | \n", "
| 7 | \n", "2005 | \n", "4901 | \n", "
| 8 | \n", "2006 | \n", "7256 | \n", "
| 9 | \n", "2007 | \n", "6767 | \n", "
| 10 | \n", "2008 | \n", "7325 | \n", "
| 11 | \n", "2009 | \n", "5164 | \n", "
| 12 | \n", "2010 | \n", "5351 | \n", "
| 13 | \n", "2011 | \n", "5341 | \n", "
| 14 | \n", "2012 | \n", "6739 | \n", "
| 15 | \n", "2013 | \n", "7525 | \n", "
| 16 | \n", "2014 | \n", "10562 | \n", "
| 17 | \n", "2015 | \n", "9706 | \n", "
| 18 | \n", "2016 | \n", "11367 | \n", "
| 19 | \n", "2017 | \n", "19574 | \n", "
| 20 | \n", "2018 | \n", "21931 | \n", "
| 21 | \n", "2019 | \n", "21615 | \n", "
| 22 | \n", "2020 | \n", "31397 | \n", "
| 23 | \n", "2021 | \n", "31034 | \n", "
| 24 | \n", "2022 | \n", "33547 | \n", "
| 25 | \n", "2023 | \n", "39863 | \n", "
| 26 | \n", "2024 | \n", "13083 | \n", "