"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Guided Hunting - Solarwinds Post Compromise\r\n",
"\r\n",
" Details...\r\n",
" \r\n",
"**Notebook Version:** 1.0 \r\n",
"**Python Version:** Python 3.7 (including Python 3.6 - AzureML) \r\n",
"**Required Packages**: kqlmagic, msticpy, pandas, numpy, matplotlib, networkx, ipywidgets, ipython, scikit_learn, dnspython, ipwhois, folium, holoviews \r\n",
"\r\n",
"**Data Sources Required**:\r\n",
"- Log Analytics \r\n",
" - Heartbeat\r\n",
" - SecurityAlert\r\n",
" - SecurityEvent\r\n",
" - DeviceProcessEvents\r\n",
" - DeviceNetworkEvents\r\n",
" - DeviceFileEvents\r\n",
" - SigninEvents\r\n",
" - AuditLogs\r\n",
" - AzureNetworkAnalytics_CL\r\n",
" \r\n",
" \r\n",
"- (Optional) \r\n",
" - VirusTotal (with API key)\r\n",
" - Alienvault OTX (with API key) \r\n",
" - IBM Xforce (with API key) \r\n",
" - CommonSecurityLog\r\n",
"\r\n",
"\r\n",
"This Notebook assists defenders in hunting for Solarwinds Post compromise Tactics , Tools and Procedures (TTPs) across different environments both on-prem and cloud data sources. \r\n",
"\r\n",
"You can read more about the attack in below technical blogs\r\n",
"\r\n",
"**References :**\r\n",
"\r\n",
"- https://msrc-blog.microsoft.com/2020/12/13/customer-guidance-on-recent-nation-state-cyber-attacks/\r\n",
"- https://www.fireeye.com/blog/threat-research/2020/12/evasive-attacker-leverages-solarwinds-supply-chain-compromises-with-sunburst-backdoor.html\r\n",
"- \r\n",
"\r\n",
"**How to use:** \r\n",
"Run the cells in this Notebook in order, at various points in the Notebook flow you will be prompted to enter or select options relevant to the scope of your triage. \r\n",
"This Notebook presumes you have Microsoft Sentinel Workspace settings and Threat Intelligence providers configured in a config file. If you do not have this in place please refer https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file to https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb and to set this up."
]
},
{
"cell_type": "markdown",
"metadata": {
"ExecuteTime": {
"end_time": "2020-12-15T21:39:44.390403Z",
"start_time": "2020-12-15T21:39:44.380860Z"
}
},
"source": [
"---\n",
"## Notebook initialization\n",
"The next cell:\n",
"- Checks for the correct Python version\n",
"- Checks versions and optionally installs required packages\n",
"- Imports the required packages into the notebook\n",
"- Sets a number of configuration options.\n",
"\n",
"This should complete without errors. If you encounter errors or warnings look at the following two notebooks:\n",
"- [TroubleShootingNotebooks](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/TroubleShootingNotebooks.ipynb)\n",
"- [ConfiguringNotebookEnvironment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)\n",
"\n",
"If you are running in the Microsoft Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:\n",
"- [Run TroubleShootingNotebooks](./TroubleShootingNotebooks.ipynb)\n",
"- [Run ConfiguringNotebookEnvironment](./ConfiguringNotebookEnvironment.ipynb)\n",
"\n",
"You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup. \n",
"There are more details about this in the `ConfiguringNotebookEnvironment` notebook and in these documents:\n",
"- [msticpy configuration](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)\n",
"- [Threat intelligence provider configuration](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"ExecuteTime": {
"end_time": "2020-12-16T17:26:20.819278Z",
"start_time": "2020-12-16T17:26:20.784441Z"
}
},
"outputs": [],
"source": [
"from pathlib import Path\r\n",
"from IPython.display import display, HTML\r\n",
"\r\n",
"REQ_PYTHON_VER=(3, 6)\r\n",
"REQ_MSTICPY_VER=(1, 0, 0)\r\n",
"\r\n",
"\r\n",
"display(HTML(\"
Starting Notebook setup...
\"))\r\n",
" \r\n",
"\r\n",
"# If not using Azure Notebooks, install msticpy with\r\n",
"# %pip install msticpy\r\n",
"from msticpy.nbtools import nbinit\r\n",
"\r\n",
"additional_packages = [\r\n",
" \"tldextract\", \"IPWhois\", \"python-whois\"\r\n",
"]\r\n",
"nbinit.init_notebook(\r\n",
" namespace=globals(),\r\n",
" additional_packages=additional_packages,\r\n",
");\r\n",
"\r\n",
"from datetime import date\r\n",
"from functools import lru_cache\r\n",
"from msticpy.nbtools.foliummap import get_map_center, get_center_ip_entities\r\n",
"from pandas import json_normalize\r\n",
"from ruamel.yaml import YAML\r\n",
"from tqdm.notebook import tqdm\r\n",
"import IPython\r\n",
"import csv\r\n",
"import dns\r\n",
"import glob\r\n",
"import io\r\n",
"import json\r\n",
"import re\r\n",
"import requests\r\n",
"import tldextract\r\n",
"import whois\r\n",
"import zipfile\r\n",
"from bokeh.plotting import figure"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"ExecuteTime": {
"end_time": "2020-12-15T22:51:35.883260Z",
"start_time": "2020-12-15T22:51:17.531950Z"
}
},
"outputs": [],
"source": [
"# See if we have a Microsoft Sentinel Workspace defined in our config file.\n",
"# If not, let the user specify Workspace and Tenant IDs\n",
"\n",
"ws_config = WorkspaceConfig()\n",
"if not ws_config.config_loaded:\n",
" ws_config.prompt_for_ws()\n",
" \n",
"qry_prov = QueryProvider(data_environment=\"AzureSentinel\")\n",
"print(\"done\")\n",
" \n",
"ti = TILookup()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Configuration\n",
"\n",
"#### `msticpyconfig.yaml` configuration File\n",
"You can configure primary and secondary TI providers and any required parameters in the `msticpyconfig.yaml` file. This is read from the current directory or you can set an environment variable (`MSTICPYCONFIG`) pointing to its location.\n",
"\n",
"To configure this file see the [ConfigureNotebookEnvironment notebook](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Authenticate to Microsoft Sentinel workspace\n",
"qry_prov.connect(ws_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"ExecuteTime": {
"end_time": "2020-12-16T18:00:53.013979Z",
"start_time": "2020-12-16T18:00:53.008322Z"
}
},
"outputs": [],
"source": [
"def get_solarwinds_queries_from_github(git_url, outputdir):\n",
" r = requests.get(git_url)\n",
"\n",
" repo_zip = io.BytesIO(r.content)\n",
"\n",
" archive = zipfile.ZipFile(repo_zip, mode=\"r\")\n",
"\n",
" # Only extract Detections and Hunting Queries Folder\n",
" for file in archive.namelist():\n",
" if file.startswith(\n",
" (\n",
" \"Azure-Sentinel-master/Detections/DeviceEvents/SolarWinds_TEARDROP_Process-IOCs.yaml\",\n",
" \"Azure-Sentinel-master/Detections/DeviceNetworkEvents/SolarWinds_SUNBURST_Network-IOCs.yaml\",\n",
" \"Azure-Sentinel-master/Detections/DeviceProcessEvents/SolarWinds_SUNBURST_Process-IOCs.yaml\",\n",
" \"Azure-Sentinel-master/Detections/DeviceFileEvents/SolarWinds_SUNBURST_&_SUPERNOVA_File-IOCs.yaml\",\n",
" \"Azure-Sentinel-master/Detections/AuditLogs/MaliciousOAuthApp_O365AttackToolkit.yaml\",\n",
" \"Azure-Sentinel-master/Detections/AuditLogs/MaliciousOAuthApp_PwnAuth.yaml\",\n",
" \"Azure-Sentinel-master/Detections/AuditLogs/UseraddedtoPrivilgedGroups.yaml\",\n",
" \"Azure-Sentinel-master/Detections/AuditLogs/ADFSDomainTrustMods.yaml\",\n",
" \"Azure-Sentinel-master/Detections/AuditLogs/RareApplicationConsent.yaml\",\n",
" \"Azure-Sentinel-master/Detections/SigninLogs/AzureAADPowerShellAnomaly.yaml\",\n",
" \"Azure-Sentinel-master/Detections/OfficeActivity/MailItemsAccessedTimeSeries.yaml\",\n",
" \"Azure-Sentinel-master/Detections/SecurityEvent/RDP_RareConnection.yaml\",\n",
" \"Azure-Sentinel-master/Detections/SecurityEvent/RDP_Nesting.yaml\",\n",
" \"Azure-Sentinel-master/Detections/SecurityEvent/UserCreatedAddedToBuiltinAdmins_1d.yaml\",\n",
" \"Azure-Sentinel-master/Hunting Queries/SecurityEvent/HostsWithNewLogons.yaml\",\n",
" \"Azure-Sentinel-master/Hunting Queries/MultipleDataSources/TrackingPrivAccounts.yaml\",\n",
" \"Azure-Sentinel-master/Hunting Queries/SecurityEvent/ProcessEntropy.yaml\",\n",
" \"Azure-Sentinel-master/Hunting Queries/SecurityEvent/RareProcbyServiceAccount.yaml\",\n",
" \"Azure-Sentinel-master/Hunting Queries/SecurityEvent/uncommon_processes.yaml\"\n",
" )\n",
" ):\n",
" archive.extract(file, path=outputdir)\n",
" print(\"Downloaded and Extracted Files successfully\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"ExecuteTime": {
"end_time": "2020-12-16T18:00:53.848269Z",
"start_time": "2020-12-16T18:00:53.834945Z"
}
},
"outputs": [],
"source": [
"def_path = Path.joinpath(Path(os.getcwd()))\r\n",
"path_wgt = widgets.Text(value=str(def_path), \r\n",
" description='Path to extract to zipped repo files: ', \r\n",
" layout=widgets.Layout(width='50%'),\r\n",
" style={'description_width': 'initial'})\r\n",
"path_wgt"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"ExecuteTime": {
"end_time": "2020-12-16T18:01:10.466863Z",
"start_time": "2020-12-16T18:00:54.639740Z"
}
},
"outputs": [],
"source": [
"# Download the Microsoft Sentinel Github repo as ZIP\n",
"azsentinel_git_url = 'https://github.com/Azure/Azure-Sentinel/archive/master.zip'\n",
"\n",
"get_solarwinds_queries_from_github(git_url=azsentinel_git_url, outputdir=path_wgt.value)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"ExecuteTime": {
"end_time": "2020-12-16T18:01:11.494383Z",
"start_time": "2020-12-16T18:01:10.838554Z"
}
},
"outputs": [],
"source": [
"QUERIES_PATH = 'Azure-Sentinel-master'\n",
"sentinel_root = Path(path_wgt.value) / QUERIES_PATH\n",
"\n",
"display(HTML(\"
Listings under Detections...\"))\n",
"%ls '{sentinel_root}/Detections/'\n",
"\n",
"display(HTML(\"