{ "cells": [ { "cell_type": "markdown", "id": "604150b3-255c-441f-8f67-c95b2f91158e", "metadata": {}, "source": [ "# CVE" ] }, { "cell_type": "markdown", "id": "9688bdc8-ceb9-4fb5-a43b-588a5f108d83", "metadata": {}, "source": [ "**Common Vulnerabilities and Exposures Identifier (CVE ID)** is a unique, alphanumeric identifier assigned by the CVE Program. Each identifier references a specific vulnerability. A CVE ID enables automation and multiple parties to discuss, share, and correlate information about a specific vulnerability, knowing they are referring to the same thing\n", "\n", "> source: [www.cve.org](https://www.cve.org/ResourcesSupport/Glossary?activeTerm=glossaryCVEID)" ] }, { "cell_type": "markdown", "id": "bb012dcd-8476-4501-8ca2-1008a08588e3", "metadata": {}, "source": [ "You can see this notebook directly via:\n", "- [GitHub](https://github.com/LimberDuck/limberduck_org_julio_7/blob/main/docs/notebooks/cve/cve.ipynb)\n", "- [Jupter nbviewer](https://nbviewer.org/github/LimberDuck/limberduck_org_julio_7/blob/main/docs/notebooks/cve/cve.ipynb)" ] }, { "cell_type": "markdown", "id": "f7c29080-90bd-4e34-bc6f-92511ed31595", "metadata": {}, "source": [ "## Generation time" ] }, { "cell_type": "code", "execution_count": 1, "id": "ff06696a-18c2-4c59-9cae-bc0dd8b7b308", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2024-04-16 05:31:46 +0000\n" ] } ], "source": [ "from datetime import datetime, timezone, timedelta\n", "\n", "timezone_offset = 0.0\n", "tzinfo = timezone(timedelta(hours=timezone_offset))\n", "generation_time = datetime.now(tzinfo).strftime('%Y-%m-%d %H:%M:%S %z')\n", "print(generation_time)" ] }, { "cell_type": "markdown", "id": "f3a4c46a-1ece-4601-9f72-90d64e12f888", "metadata": {}, "source": [ "## Creative Commons" ] }, { "cell_type": "markdown", "id": "33983601-bf85-4ba0-babc-5e3a69bc5ef4", "metadata": {}, "source": [ "This notebook and generated diagrams are released with [Creative Commons liecense (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/deed.en).\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 2, "id": "17811d3a-f62b-4c35-9bad-75fcdc9e9cf5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cc.xlarge.png\n", "by.xlarge.png\n" ] } ], "source": [ "import requests\n", "import urllib3\n", "\n", "urllib3.disable_warnings()\n", "\n", "urls = ['https://mirrors.creativecommons.org/presskit/icons/cc.xlarge.png',\n", " 'https://mirrors.creativecommons.org/presskit/icons/by.xlarge.png']\n", "for url in urls:\n", " file_name = url.split(\"/\")[-1:][0]\n", " print(file_name)\n", "\n", " file = requests.get(url, verify=False)\n", " open(file_name, 'wb').write(file.content)" ] }, { "cell_type": "markdown", "id": "f0d55e25-0b30-4377-95cf-20f471fcaf21", "metadata": {}, "source": [ "## CVE data downloading" ] }, { "cell_type": "markdown", "id": "b29a1112-344a-4015-91d1-c1ee0aa63629", "metadata": {}, "source": [ "All CVE IDs are taken from [cve.mitre.org/data/downloads/index.html](https://cve.mitre.org/data/downloads/index.html)" ] }, { "cell_type": "code", "execution_count": 3, "id": "239ee776-f15c-43d4-a8f5-d1ca251f0f37", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "allitems.xml.Z\n" ] } ], "source": [ "url = 'https://cve.mitre.org/data/downloads/allitems.xml.Z'\n", "file_name = url.split(\"/\")[-1:][0]\n", "print(file_name)" ] }, { "cell_type": "code", "execution_count": 4, "id": "50f24f67-75e2-4d85-bf23-47182cbf46f3", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "69886215" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import requests\n", "import urllib3\n", "\n", "urllib3.disable_warnings()\n", "\n", "file = requests.get(url, verify=False)\n", "open(file_name, 'wb').write(file.content)" ] }, { "cell_type": "code", "execution_count": 5, "id": "e02dc4ee", "metadata": {}, "outputs": [], "source": [ "import unlzw3\n", "from pathlib import Path\n", "\n", "uncompressed_data = unlzw3.unlzw(Path(file_name))\n" ] }, { "cell_type": "code", "execution_count": 6, "id": "8b59663f", "metadata": {}, "outputs": [], "source": [ "with open(file_name[:-2], 'wb') as file:\n", " file.write(uncompressed_data)" ] }, { "cell_type": "code", "execution_count": 7, "id": "d2ff78a8", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "allitems.xml\n" ] } ], "source": [ "import glob\n", "\n", "file_name = glob.glob('*.xml')[-1]\n", "print(file_name)" ] }, { "cell_type": "markdown", "id": "cb262f32-6398-44c9-a365-d5e1b47dfcd8", "metadata": {}, "source": [ "## CVE data parsing" ] }, { "cell_type": "code", "execution_count": 8, "id": "69608e9f-cbad-40db-85f2-48d25d1aa381", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " number year\n", "0 CVE-1999-0001 1999\n", "1 CVE-1999-0002 1999\n", "2 CVE-1999-0003 1999\n", "3 CVE-1999-0004 1999\n", "4 CVE-1999-0005 1999\n", "... ... ...\n", "311256 CVE-2024-30266 2024\n", "311257 CVE-2024-30267 2024\n", "311258 CVE-2024-30268 2024\n", "311259 CVE-2024-30269 2024\n", "311260 CVE-2024-30270 2024\n", "\n", "[311261 rows x 2 columns]\n" ] } ], "source": [ "import pandas as pd \n", "import xml.etree.ElementTree as et \n", "\n", "tree = et.parse(file_name)\n", "root = tree.getroot()\n", "df_cols = [\"number\", \"year\"]\n", "rows = []\n", "\n", "for item in root:\n", " item_name = item.attrib.get(\"name\")\n", " item_year = item_name[4:8]\n", " rows.append({\"number\": item_name, \"year\": item_year})\n", "\n", "df = pd.DataFrame(rows, columns = df_cols)\n", "\n", "print(df)" ] }, { "cell_type": "code", "execution_count": 9, "id": "ecbe6644-37e1-4747-b8cc-3181570bfb1e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " | year | \n", "number | \n", "
---|---|---|
1 | \n", "1999 | \n", "1579 | \n", "
2 | \n", "2000 | \n", "1243 | \n", "
3 | \n", "2001 | \n", "1573 | \n", "
4 | \n", "2002 | \n", "2436 | \n", "
5 | \n", "2003 | \n", "1603 | \n", "
6 | \n", "2004 | \n", "2779 | \n", "
7 | \n", "2005 | \n", "4901 | \n", "
8 | \n", "2006 | \n", "7256 | \n", "
9 | \n", "2007 | \n", "6767 | \n", "
10 | \n", "2008 | \n", "7325 | \n", "
11 | \n", "2009 | \n", "5164 | \n", "
12 | \n", "2010 | \n", "5351 | \n", "
13 | \n", "2011 | \n", "5341 | \n", "
14 | \n", "2012 | \n", "6739 | \n", "
15 | \n", "2013 | \n", "7525 | \n", "
16 | \n", "2014 | \n", "10562 | \n", "
17 | \n", "2015 | \n", "9706 | \n", "
18 | \n", "2016 | \n", "11367 | \n", "
19 | \n", "2017 | \n", "19574 | \n", "
20 | \n", "2018 | \n", "21931 | \n", "
21 | \n", "2019 | \n", "21615 | \n", "
22 | \n", "2020 | \n", "31397 | \n", "
23 | \n", "2021 | \n", "31034 | \n", "
24 | \n", "2022 | \n", "33547 | \n", "
25 | \n", "2023 | \n", "39863 | \n", "
26 | \n", "2024 | \n", "13083 | \n", "