{ "cells": [ { "cell_type": "markdown", "source": [ "# Getting Started with Azure ML Notebooks and Microsoft Sentinel\n", "\n", "---\n", "\n", "## Pre-requisites\n", "- Log Analytics *Reader* permissions on the Microsoft Sentinel workspace\n", "- Python 3.8 notebook kernel (`Python 3.8 - Azure ML`)\n", "\n", "# Contents\n", "\n", "1. **Introduction**
\n", " 1.1 What is a Jupyter notebook?
\n", " 1.2 Running code in notebooks

\n", "2. **Initializing the notebook and MSTICPy**
\n", "3. **Querying Data from Microsoft Sentinel**
\n", " 3.1 Verifying Microsoft Sentinel settings
\n", " 3.2 (Optional) Configure your Azure Cloud
\n", " 3.3 Load a QueryProvider for Microsoft Sentinel
\n", " 3.4 Authenticate to the Microsoft Sentinel workspace
\n", " 3.5 Test your connection using a MSTICPy built-in Microsoft Sentinel query

\n", "4. **Configure and test external data providers (VirusTotal and Maxmind GeoLite2)**
\n", " 4.1 (Optional) Configure Azure Key Vault to store secrets
\n", " 4.2 Testing VirusTotal Lookup
\n", " 4.3 Testing IP geolocation lookup with Maxmind GeoLite2

\n", "5. **Conclusion and Next Steps**
\n", "6. **Further Resources**
\n", "7. **FAQs - Frequently Asked Questions**\n" ], "metadata": {} }, { "cell_type": "markdown", "source": [ "\n", "---\n", "\n", "# 1. Introduction\n", "\n", "This notebook takes you through the basics needed to get started with Azure Machine Learning (ML) Notebooks and Microsoft Sentinel.\n", "\n", "
\n", "

Warning. Due to rendering issues in Azure Machine Learning, we strongly recommend running this notebook in Jupyter Lab or VSCode.

\n", "
\n", "To do this:\n", "\n", "The MSTICPy settings editor uses notebook widgets, which are not\n", "fully supported in AML notebooks.\n", "
\n", "
\n", "\n", "It focuses on getting things set up and basic steps to query data.\n", "\n", "After you've finished running this notebook you can go on to look at the following notebooks:\n", "\n", "- **A Tour of Cybersec notebook features** - this takes you through some of the basic\n", " features for CyberSec investigation/hunting available to you in notebooks.\n", "- **Configuring your environment** - this covers all of the configuration options for \n", " accessing external cybersec resources\n", "\n", "\n", "Each topic includes 'learn more' sections to provide you with the resource to deep\n", "dive into each of these topics. We encourage you to work through the notebook from start\n", "to finish.\n", "\n", "Use these documents for background and more details about the steps in this notebook:\n", "- [Use Jupyter notebooks to hunt for security threats](https://docs.microsoft.com/azure/sentinel/notebooks)\n", "- [Tutorial: Get started with Jupyter notebooks and MSTICPy in Microsoft Sentinel](https://docs.microsoft.com/azure/sentinel/notebook-get-started)\n", "\n", "The second document follows the steps in this notebook. Note that this notebook uses a simplified interface for configuring settings rather that the MSTICPy configuration tool described in the documents.\n", "\n", "\n", "
\n", "

Please run the the code cells in sequence. Skipping cells will result in errors.

\n", "
\n", "

If you encounter any unexpected errors or warnings please see the FAQ at the end of this notebook.

\n", "
\n", "
\n" ], "metadata": {} }, { "cell_type": "markdown", "source": [ "---\n", "\n", "## 1.1. What is a Jupyter notebook?\n", "\n", "
\n", "If you're familiar with notebooks, skip this section and go to \"2. Initializing\n", "the notebook and MSTICPy\" section.\n", "
\n", "
\n", "\n", "You are currently reading a Jupyter notebook. [Jupyter](http://jupyter.org/) is an interactive\n", "development and data manipulation environment presented in a browser.\n", "\n", "A Jupyter notebook is a document\n", "made up of cells that contain interactive code, alongside that code's output,\n", "and other items such as text and images (what you are looking at now is a cell of *Markdown* text).\n", "\n", "
\n", " More...\n", "The name, Jupyter, comes from the core supported programming languages that it supports: **Ju**lia, **Pyt**hon, and **R**.\n", "While you can use any of these languages (and others such as Powershell) we are going to use Python in this notebook.\n", "\n", "The majority of the notebooks on the Microsoft Sentinel GitHub repo\n", "are written in Python. Whilst there are pros, and cons to each language, Python is a well-established\n", "language that has a large number of materials and libraries well suited for\n", "data analysis and security investigation, making it ideal for our needs.\n", "
\n", "\n", "To use a Jupyter notebook you need a Jupyter server that will render the notebook and execute the code within it.\n", "This can take the form of a local [Jupyter installation](https://pypi.org/project/jupyter/),\n", "or a remotely hosted version such as \n", "[Azure Machine Learning Notebooks](https://docs.microsoft.com/azure/machine-learning/how-to-run-jupyter-notebooks). \n", "\n", "
\n", " Learn more...\n", " \n", "
\n", "\n", "
\n", "\n" ], "metadata": {} }, { "cell_type": "markdown", "source": [ "---\n", "\n", "## 1.2 Running code in notebooks\n", "\n", "The **cell** below is a code cell (note that it looks different from the\n", "cell you are reading). The current cell is known as a *Markdown* cell\n", "and lets you write text (including HTML) and include static images.\n", "\n", "Select the code cell (using mouse or cursor keys) below.\n", "Once selected, you can execute the code in it by clicking the \"Play\" button in the cell, or by pressing Shift+Enter.\n", "\n", "

\n", "Tip: You can identify which cells are code cells by selecting them.
\n", "In Azure ML notebooks and VSCode, code cells have a larger border\n", "on the left side with a \"Play\" button to execute the cell.
\n", "In other notebook environments code and markdown cells will have\n", "different styles but it's usually easy to distinguish them.\n", "

\n" ], "metadata": {} }, { "cell_type": "code", "source": [ "# This is our first code cell, it contains basic Python code.\n", "# You can run a code cell by selecting it and clicking\n", "# the Run button (to the left of the cell), or by pressing Shift + Enter.\n", "# Any output from the code will be displayed directly below it.\n", "print(\"Congratulations, you just ran this code cell\")\n", "y = 2 + 2\n", "print(\"2 + 2 =\", y)" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": "Congratulations, you just ran this code cell\n2 + 2 = 4\n" } ], "execution_count": 3, "metadata": { "gather": { "logged": 1684963940618 } } }, { "cell_type": "markdown", "source": [ "Variables set within a code cell persist between cells meaning you can chain cells together.\n", "\n", "In this example we're using the value of `y` from the previous cell." ], "metadata": {} }, { "cell_type": "code", "source": [ "# Note that output from the last line of a cell is automatically\n", "# sent to the output cell, without needing the print() function.\n", "y + 2" ], "outputs": [ { "output_type": "execute_result", "execution_count": 4, "data": { "text/plain": "6" }, "metadata": {} } ], "execution_count": 4, "metadata": { "gather": { "logged": 1684963944062 } } }, { "cell_type": "markdown", "source": [ "Now that you understand the basics we can move onto more complex code.\n", "\n", "\n", "
\n", " Learn more about notebooks...\n", "
\n" ], "metadata": {} }, { "cell_type": "markdown", "source": [ "---\n", "\n", "# 2. Initializing the notebook and MSTICPy\n", "\n", "
\n", " What are Python packages?\n", "To avoid having to type (or paste) a lot of complex and repetitive code into\n", "notebook cells, most notebooks rely on third party libraries (known in the Python\n", "world as \"packages\").\n", "\n", "Before you can use a package in your notebook, you need to do two things:\n", "\n", "- install the package (although the Azure ML Compute has most common packages pre-installed)\n", "- import the package (or some part of the package - usually a module/file, a function or a class)\n", "
\n", "\n", "## MSTICPy\n", "\n", "**MSTICPy** (pronounced miss-tick-pie) is a Python package of CyberSecurity tools for data retrieval, analysis, enrichment and visualization.\n", "\n", "## Initializing notebooks\n", "\n", "At the start of most Microsoft Sentinel notebooks you will see an initialization cell like the one below.\n", "This cell is specific to the MSTICPy initialization:\n", "\n", "- it defines the minimum versions for Python and MSTICPy needed for this notebook\n", "- it then imports and runs the `init_notebook` function.\n", "\n", "
\n", " More about init_notebook...\n", "

\n", "`init_notebook` does some of the tedious work of importing other packages, \n", "checking configuration (we'll get to configuration in a moment) and, optionally,\n", "installing other required packages.

\n", "
\n", "
\n", "\n", "
\n", "Notes: \n", "

1. Don't be alarmed if you see configuration warnings (such as \"Missing msticpyconfig.yaml\").
\n", "We haven't configured anything yet, so this is expected.

\n", "

2. You may also see some warnings about package version conflicts. It is usually safe\n", "to ignore these.

\n", "
\n", "\n", "The `%pip install` line ensures that the latest version of msticpy is installed." ], "metadata": {} }, { "cell_type": "code", "source": [ "# import some modules needed in this cell\n", "from IPython.display import display, HTML\n", "\n", "display(HTML(\"Checking upgrade to latest msticpy version\"))\n", "%pip install --upgrade --quiet msticpy\\[sentinel\\]\n", "\n", "\n", "REQ_PYTHON_VER=\"3.8\"\n", "REQ_MSTICPY_VER=\"1.5.2\"\n", "\n", "# initialize msticpy\n", "import msticpy\n", "msticpy.init_notebook(namespace=globals());" ], "outputs": [ { "output_type": "display_data", "data": { "text/plain": "", "text/html": "Checking upgrade to latest msticpy version" }, "metadata": {} }, { "output_type": "stream", "name": "stdout", "text": "Note: you may need to restart the kernel to use updated packages.\n" }, { "output_type": "display_data", "data": { "text/plain": "", "text/html": "

Starting notebook pre-checks...

" }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": "", "text/html": "Checking Python kernel version..." }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": "", "text/html": "Info: Python kernel version 3.8.5 - OK
" }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": "", "text/html": "Checking msticpy version...
" }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": "", "text/html": "Info: msticpy version 2.4.0 (>= 1.5.2) - OK
" }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": "", "text/html": "

Notebook pre-checks complete.

" }, "metadata": {} } ], "execution_count": 5, "metadata": { "gather": { "logged": 1684963956644 } } }, { "cell_type": "markdown", "source": [ "---\n", "\n", "# 3. Querying Data from Microsoft Sentinel\n", "\n", "Once we've done this basic initialization step,\n", "we need to make sure we have configuration to tell MSTICPy how to connect\n", "to your workspace.\n", "\n", "This configuration is stored in a configuration file (`msticpyconfig.yaml`).
\n", "\n", "
\n", " Learn more...\n", "

\n", " Although you don't need to know these details now, you can find more information here:\n", "

\n", " \n", "

If you need a more complete walk-through of configuration, we have a separate notebook to help you:

\n", "
    \n", "
  • Configuring Notebook Environment
    • \n", "
  • And for the ultimate walk-through of how to configure all your `msticpyconfig.yaml` settings\n", " see the MPSettingsEditor notebook
    • \n", "
  • The Azure-Sentinel-Notebooks GitHub repo also contains an template `msticpyconfig.yaml`, with commented-out sections\n", " that may also be helpful in finding your way around the settings if you want to dig into things\n", " by hand.
    • \n", "
\n", "
\n", "
\n", "\n", "---" ], "metadata": {} }, { "cell_type": "markdown", "source": [ "## 3.1 Verifying Microsoft Sentinel settings\n", "\n", "When you launched this notebook from Microsoft Sentinel a basic configuration file - `config.json` -\n", "was copied to your workspace folder.
\n", "You should be able to see this file in the file browser to the left.
\n", "This file contains details about your Microsoft Sentinel workspace but has\n", "no configuration settings for other external services that we need.\n", "\n", "If you didn't have a `msticpyconfig.yaml` file in your workspace folder the \n", "`init_notebook` function should have created one for you and populated it\n", "with the Microsoft Sentinel workspace data taken from your config.json.\n", "\n", "

Tip:\n", "If you do not see a \"msticpyconfig.yaml\" file in your user folder, click the refresh button
\n", "at the top of the file browser.\n", "

\n", "\n", "We can check this now by display the settings.\n", "\n", "
\n", " Multiple Microsoft Sentinel workspaces...\n", "

If you have multiple Microsoft Sentinel workspaces, you can add\n", " them in the following configuration cell.

\n", "

You can choose to keep one as the default or just delete this entry\n", " if you always want to name your workspaces explicitly when you \n", " connect.\n", "

\n", "
" ], "metadata": {} }, { "cell_type": "code", "source": [ "import msticpy\n", "from msticpy.config import MpConfigFile, MpConfigEdit\n", "import os\n", "import json\n", "from pathlib import Path\n", "\n", "mp_conf = \"msticpyconfig.yaml\"\n", "\n", "# check if MSTICPYCONFIG is already an env variable\n", "mp_env = os.environ.get(\"MSTICPYCONFIG\")\n", "mp_conf = mp_env if mp_env and Path(mp_env).is_file() else mp_conf\n", "\n", "if not Path(mp_conf).is_file():\n", " print(\n", " \"No msticpyconfig.yaml was found!\",\n", " \"Please check that there is a config.json file in your workspace folder.\",\n", " \"If this is not there, go back to the Microsoft Sentinel portal and launch\",\n", " \"this notebook from there.\",\n", " sep=\"\\n\"\n", " )\n", "else:\n", " mpedit = MpConfigEdit(mp_conf)\n", " mpconfig = MpConfigFile(mp_conf)\n", " print(f\"Configured Sentinel workspaces: {json.dumps(mpconfig.settings, indent=4)}\")\n", "\n", "msticpy.settings.refresh_config()" ], "outputs": [ { "output_type": "display_data", "data": { "text/plain": "Label(value='Loading. Please wait.')", "application/vnd.jupyter.widget-view+json": { "version_major": 2, "version_minor": 0, "model_id": "4acdca79a6604040a526248a61e076b9" } }, "metadata": {} }, { "output_type": "stream", "name": "stdout", "text": "Configured Sentinel workspaces: {\n \"AzureSentinel\": {\n \"Workspaces\": {\n \"Default\": {\n \"ResourceGroup\": \"zhzhaopitest\",\n \"SubscriptionId\": \"b297e6df-ac0e-4f46-87aa-7cdafe177f4b\",\n \"TenantId\": \"72f988bf-86f1-41af-91ab-2d7cd011db47\",\n \"WorkspaceId\": \"60373d82-0640-4926-8e75-4e76ad7afe49\"\n },\n \"zhzhaoasi\": {\n \"ResourceGroup\": \"zhzhaopitest\",\n \"SubscriptionId\": \"b297e6df-ac0e-4f46-87aa-7cdafe177f4b\",\n \"TenantId\": \"72f988bf-86f1-41af-91ab-2d7cd011db47\",\n \"WorkspaceId\": \"60373d82-0640-4926-8e75-4e76ad7afe49\"\n }\n }\n }\n}\n" } ], "execution_count": 6, "metadata": { "gather": { "logged": 1684963985225 } } }, { "cell_type": "markdown", "source": [ "At this stage you should only see two entries in the `Azure Sentinel\\Workspaces` section:\n", "\n", "- An entry with the name of your Microsoft Sentinel workspace\n", "- An entry named \"Default\" with the same settings." ], "metadata": {} }, { "cell_type": "markdown", "source": [ "## 3.2 (Optional) Configure your Azure Cloud\n", "\n", "If you are running in a sovereign or government cloud (i.e. not the Azure global cloud)\n", "you must set up Azure functions to use the correct authentication and\n", "resource management authorities.\n", "\n", "

Note:\n", "This is not required if using the Azure Global cloud (most common)\n", "and you can skip this step.

\n", "\n", "If the domain of your Microsoft Sentinel or Azure Machine learning does\n", "not end with '.azure.com' you should set the appropriate cloud\n", "for your organization.\n", "\n", "If you change to a different cloud, hit **Update** and **Save Settings** to write\n", "the changes to your configuration file." ], "metadata": {} }, { "cell_type": "code", "source": [ "display(mpedit)\n", "mpedit.set_tab(\"Azure\")\n" ], "outputs": [ { "output_type": "display_data", "data": { "text/plain": "VBox(children=(Tab(children=(VBox(children=(Label(value='Microsoft Sentinel workspace settings'), HBox(childre…", "application/vnd.jupyter.widget-view+json": { "version_major": 2, "version_minor": 0, "model_id": "8edf53ee5ad247a180dd31cc1da28fef" } }, "metadata": {} } ], "execution_count": 10, "metadata": { "gather": { "logged": 1684964030941 } } }, { "cell_type": "markdown", "source": [ "## 3.3 Load a QueryProvider for Microsoft Sentinel\n", "\n", "To start, we are going to load up a `QueryProvider`\n", "for Microsoft Sentinel. The `QueryProvider` is the object you use to\n", "querying data from MS Sentinel and make it available to view and analyze in the notebook.\n", "There are two steps to do this:\n", "1. Create the `QueryProvider`\n", "2. run the `connect` function to authenticate to the Sentinel workspace.\n", "\n", "
Note:\n", "If you see a warning \"Runtime dependency of PyGObject is missing\" when loading the
\n", "Microsoft Sentinel driver, please see the FAQ section at the end of this notebook.
\n", "The warning does not impact any functionality of the notebooks.\n", "
\n", "
\n", "
\n", " More about query providers...\n", "

Query results are always returned as pandas DataFrames.

\n", "

If you are new\n", "to using pandas look at the Introduction to Pandas section at in\n", "the A Tour of Cybersec notebook features notebook.

\n", "
\n", "

\n", "The query provider supports other data sources, as well as Microsoft Sentinel.

\n", "

\n", "Other data sources supported by the `QueryProvider` class include Microsoft Defender for Endpoint,\n", "Splunk, Microsoft Graph API, Azure Resource Graph but these are not covered here.\n", "

\n", "

\n", "Most query providers come with a range of built-in queries\n", "for common data operations. You can also a query provider to run custom queries against\n", "Microsoft Sentinel data.\n", "

\n", "
\n", "Once you've loaded a QueryProvider you'll normally need to authenticate\n", "to the data source (in this case Microsoft Sentinel).\n", "\n", "
\n", "
\n" ], "metadata": {} }, { "cell_type": "code", "source": [ "# Refresh any config items that might have been saved\n", "# to the msticpyconfig in the previous steps.\n", "msticpy.settings.refresh_config()\n", "\n", "# Initialize a QueryProvider for Microsoft Sentinel\n", "qry_prov = QueryProvider(\"AzureSentinel\")" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": "Please wait. Loading Kqlmagic extension..." }, { "output_type": "display_data", "data": { "text/plain": "", "application/javascript": "try {IPython.notebook.kernel.reconnect();} catch(err) {;}" }, "metadata": {} }, { "output_type": "stream", "name": "stdout", "text": "done\n" } ], "execution_count": 8, "metadata": { "gather": { "logged": 1684963995696 } } }, { "cell_type": "markdown", "source": [ "## 3.4 Authenticate to the Microsoft Sentinel workspace\n", "\n", "Next we need to authenticate.\n", "\n", "The code cell immediately following this section will start the authentication process.\n", "\n", "In Azure ML notebooks the authentication will default to using the credentials\n", "you used to authentication to the Azure ML workspace.\n", "\n", "More information:\n", "\n", "
\n", " Alternative authentication options\n", "Instead of using the Azure ML credentials, you can opt to use\n", "one of the following:\n", "
    \n", "
  • Device authentication
    • \n", "
  • Azure CLI credentials
    • \n", "
\n", "\n", "

Device authentication uses a unique code generated on your client\n", "as an additional authentication factor. When prompted, you copy\n", "the code, open a browser to http://microsoft.com/devicelogin and paste\n", "it in. Then follow the interactive authentication flow.

\n", "\n", "Azure CLI authentication requires you to logon (in the notebook or \n", "a terminal) before authenticating to Microsoft Sentinel\n", "
az login
\n", " \n", "

You can change the authentication option used when calling \"connect\"\n", "with the following.
\n", "To force Device authentication add the following parameter\n", "to the connect call\n", "

\n",
        "qry_prov.connect(ws_config, mp_az_auth=False)\n",
        "
\n", "

\n", "

\n", "To use Azure CLI authentication:\n", "

\n",
        "qry_prov.connect(ws_config, mp_az_auth=[\"cli\"])\n",
        "
\n", "

\n", "
\n", "\n", "\n", "
\n", " Using WorkspaceConfig\n", "Loading WorkspaceConfig with no parameters will use the details\n", "of your \"Default\" workspace (see the Configuring Microsoft Sentinel settings section earlier)
\n", "\n", "If you want to connect to a specific workspace use this syntax:
\n", " 
ws_config = WorkspaceConfig(workspace=\"WorkspaceName\")
\n", "'WorkspaceName' should be one of the workspaces defined in msticpyconfig.yaml\n", "
\n" ], "metadata": {} }, { "cell_type": "code", "source": [ "# Get the default Microsoft Sentinel workspace details from msticpyconfig.yaml\n", "\n", "ws_config = WorkspaceConfig()\n", " \n", "# Connect to Microsoft Sentinel with our QueryProvider and config details\n", "qry_prov.connect(ws_config)" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": "Connecting... " }, { "output_type": "display_data", "data": { "text/plain": "", "text/html": "\n \n
\n \n \n
\n\n \n\n " }, "metadata": {} }, { "output_type": "stream", "name": "stdout", "text": "connected\n" } ], "execution_count": 9, "metadata": { "gather": { "logged": 1684964012426 } } }, { "cell_type": "markdown", "source": [ "## 3.5 Test your connection using a MSTICPy built-in Microsoft Sentinel query\n", "\n", "To explore queries in more detail see the **A Tour of CyberSec Notebook Features** notebook.\n" ], "metadata": {} }, { "cell_type": "code", "source": [ "# The time parameters are taken from the qry_prov.query_prov time settings\n", "# attribute, which provides the default query time range. You can\n", "# change interactively this by running qry_prov.query_time.\n", "alerts_df = qry_prov.SecurityAlert.list_alerts(start=qry_prov.query_time.start)\n", "\n", "if alerts_df.empty:\n", " md(\"The query returned no rows for this time range. You might want to increase the time range\")\n", "\n", "# display first 5 rows of any results\n", "alerts_df.head() # If you have no data you will just see the column headings displayed" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": "Authenticating to Azure.\n" }, { "output_type": "execute_result", "execution_count": 8, "data": { "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
TenantIdTimeGeneratedAlertDisplayNameAlertNameSeverityDescriptionProviderNameVendorNameVendorOriginalIdSystemAlertIdResourceIdSourceComputerIdAlertTypeConfidenceLevelConfidenceScoreIsIncidentStartTimeUtcEndTimeUtcProcessingEndTimeRemediationStepsExtendedPropertiesEntitiesSourceSystemWorkspaceSubscriptionIdWorkspaceResourceGroupExtendedLinksProductNameProductComponentNameAlertLinkStatusCompromisedEntityTacticsType
08ecf8077-cf51-4820-aadd-14040956f35d2021-12-21 20:01:15.205000+00:00Sign-in from an unfamiliar locationSign-in from an unfamiliar locationLowThis indicates suspicious login by JeffL to Amsterdam,Noord-Holland,Netherlands from IP: 117.30....IPCMicrosoft088efa1064dda0c95843406b3b0326a47854652addd94a236c411a949624776d183185f6-0825-6fe3-1f4b-96235a6592f7UnfamiliarLocationNaNFalse2021-12-21 19:00:51.129000+00:002021-12-21 19:58:00+00:002021-12-21 20:01:14.896000+00:00{\\r\\n \"IpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": \"72f9...[\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft...DetectionAzure Active Directory Identity ProtectionNewExploitationSecurityAlert
18ecf8077-cf51-4820-aadd-14040956f35d2021-12-21 20:01:15.556000+00:00Sign-in from an anonymous IPSign-in from an anonymous IPLowThis indicates suspicious login by JeffL to Amsterdam,Noord-Holland,Netherlands from IP: 117.30....IPCMicrosofta0aa8f7044c258464916a4fb9ae02d9752b374a64bb9ede694e977e22622ba478ca95a45-e423-f834-cbc4-fb1a829a0205AnonymousLoginNaNFalse2021-12-21 19:10:44.195000+00:002021-12-21 19:58:00+00:002021-12-21 20:01:14.896000+00:00{\\r\\n \"IpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": \"72f9...[\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft...DetectionAzure Active Directory Identity ProtectionNewExploitationSecurityAlert
28ecf8077-cf51-4820-aadd-14040956f35d2021-12-21 20:01:15.077000+00:00Impossible travel to atypical locationsImpossible travel to atypical locationsLowThis indicates suspicious login by JeffL to Amsterdam,Noord-Holland,Netherlands from IP: 117.30....IPCMicrosoft4ae272875012e62d309987fbe6994fd9b8c4d0b2b849b93ce0b604f258822ea1d875eb38-8d70-a8c1-ca2b-6bdfdb3c57b7ImpossibleTravelNaNFalse2021-12-21 19:23:24.852000+00:002021-12-21 19:58:00+00:002021-12-21 20:01:14.896000+00:00{\\r\\n \"IpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": \"72f9...[\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft...DetectionAzure Active Directory Identity ProtectionNewExploitationSecurityAlert
38ecf8077-cf51-4820-aadd-14040956f35d2021-12-21 20:01:16.232000+00:00Suspicious inbox forwardingSuspicious inbox forwardingLowThe user JeffL@seccxpninja.onmicrosoft.com created or updated an inbox forwarding rule that forw...MCASMicrosoft7dff714321c5ef88a13ddbd2674590f98e2c3b0f7599caca792061116a39feb7f220de35-08c6-a363-15e2-af898e933755MCAS_ALERT_ANUBIS_INBOX_FORWARDINGNaNFalse2021-12-21 20:02:00+00:002021-12-21 20:49:10.838000+00:002021-12-21 20:01:14.896000+00:00{\\r\\n \"DummyIpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": ...[\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft...DetectionMicrosoft Cloud App SecurityNewExfiltrationSecurityAlert
48ecf8077-cf51-4820-aadd-14040956f35d2021-12-21 20:01:15.780000+00:00Mass deleteMass deleteLowThe user JeffL@seccxpninja.onmicrosoft.com deleted more than 9,448 unique objects in a single se...MCASMicrosoft519f06a014d7fbe11c449e4e1747153c0e0a8f744a53f7d4d6fe1d9b1385479b86e1d88c-db02-1664-a6b9-1a2dd8e0ab47MCAS_ALERT_ANUBIS_DETECTION_REPEATED_ACTIVITY_DELETENaNFalse2021-12-21 20:02:00+00:002021-12-21 20:59:08.871000+00:002021-12-21 20:01:14.896000+00:00{\\r\\n \"DummyIpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": ...[\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft...DetectionMicrosoft Cloud App SecurityNewExecutionSecurityAlert
\n
", "text/plain": " TenantId TimeGenerated \\\n0 8ecf8077-cf51-4820-aadd-14040956f35d 2021-12-21 20:01:15.205000+00:00 \n1 8ecf8077-cf51-4820-aadd-14040956f35d 2021-12-21 20:01:15.556000+00:00 \n2 8ecf8077-cf51-4820-aadd-14040956f35d 2021-12-21 20:01:15.077000+00:00 \n3 8ecf8077-cf51-4820-aadd-14040956f35d 2021-12-21 20:01:16.232000+00:00 \n4 8ecf8077-cf51-4820-aadd-14040956f35d 2021-12-21 20:01:15.780000+00:00 \n\n AlertDisplayName \\\n0 Sign-in from an unfamiliar location \n1 Sign-in from an anonymous IP \n2 Impossible travel to atypical locations \n3 Suspicious inbox forwarding \n4 Mass delete \n\n AlertName Severity \\\n0 Sign-in from an unfamiliar location Low \n1 Sign-in from an anonymous IP Low \n2 Impossible travel to atypical locations Low \n3 Suspicious inbox forwarding Low \n4 Mass delete Low \n\n Description \\\n0 This indicates suspicious login by JeffL to Amsterdam,Noord-Holland,Netherlands from IP: 117.30.... \n1 This indicates suspicious login by JeffL to Amsterdam,Noord-Holland,Netherlands from IP: 117.30.... \n2 This indicates suspicious login by JeffL to Amsterdam,Noord-Holland,Netherlands from IP: 117.30.... \n3 The user JeffL@seccxpninja.onmicrosoft.com created or updated an inbox forwarding rule that forw... \n4 The user JeffL@seccxpninja.onmicrosoft.com deleted more than 9,448 unique objects in a single se... \n\n ProviderName VendorName \\\n0 IPC Microsoft \n1 IPC Microsoft \n2 IPC Microsoft \n3 MCAS Microsoft \n4 MCAS Microsoft \n\n VendorOriginalId \\\n0 088efa1064dda0c95843406b3b0326a47854652addd94a236c411a949624776d \n1 a0aa8f7044c258464916a4fb9ae02d9752b374a64bb9ede694e977e22622ba47 \n2 4ae272875012e62d309987fbe6994fd9b8c4d0b2b849b93ce0b604f258822ea1 \n3 7dff714321c5ef88a13ddbd2674590f98e2c3b0f7599caca792061116a39feb7 \n4 519f06a014d7fbe11c449e4e1747153c0e0a8f744a53f7d4d6fe1d9b1385479b \n\n SystemAlertId ResourceId SourceComputerId \\\n0 183185f6-0825-6fe3-1f4b-96235a6592f7 \n1 8ca95a45-e423-f834-cbc4-fb1a829a0205 \n2 d875eb38-8d70-a8c1-ca2b-6bdfdb3c57b7 \n3 f220de35-08c6-a363-15e2-af898e933755 \n4 86e1d88c-db02-1664-a6b9-1a2dd8e0ab47 \n\n AlertType ConfidenceLevel \\\n0 UnfamiliarLocation \n1 AnonymousLogin \n2 ImpossibleTravel \n3 MCAS_ALERT_ANUBIS_INBOX_FORWARDING \n4 MCAS_ALERT_ANUBIS_DETECTION_REPEATED_ACTIVITY_DELETE \n\n ConfidenceScore IsIncident StartTimeUtc \\\n0 NaN False 2021-12-21 19:00:51.129000+00:00 \n1 NaN False 2021-12-21 19:10:44.195000+00:00 \n2 NaN False 2021-12-21 19:23:24.852000+00:00 \n3 NaN False 2021-12-21 20:02:00+00:00 \n4 NaN False 2021-12-21 20:02:00+00:00 \n\n EndTimeUtc ProcessingEndTime \\\n0 2021-12-21 19:58:00+00:00 2021-12-21 20:01:14.896000+00:00 \n1 2021-12-21 19:58:00+00:00 2021-12-21 20:01:14.896000+00:00 \n2 2021-12-21 19:58:00+00:00 2021-12-21 20:01:14.896000+00:00 \n3 2021-12-21 20:49:10.838000+00:00 2021-12-21 20:01:14.896000+00:00 \n4 2021-12-21 20:59:08.871000+00:00 2021-12-21 20:01:14.896000+00:00 \n\n RemediationSteps \\\n0 \n1 \n2 \n3 \n4 \n\n ExtendedProperties \\\n0 {\\r\\n \"IpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": \"72f9... \n1 {\\r\\n \"IpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": \"72f9... \n2 {\\r\\n \"IpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": \"72f9... \n3 {\\r\\n \"DummyIpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": ... \n4 {\\r\\n \"DummyIpAddress\": \"117.30.165.58\",\\r\\n \"FusionSyntheticAlert\": \"true\",\\r\\n \"TenantId\": ... \n\n Entities \\\n0 [\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft... \n1 [\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft... \n2 [\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft... \n3 [\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft... \n4 [\\r\\n {\\r\\n \"$id\": \"3\",\\r\\n \"Name\": \"JeffL\",\\r\\n \"UPNSuffix\": \"seccxpninja.onmicrosoft... \n\n SourceSystem WorkspaceSubscriptionId WorkspaceResourceGroup ExtendedLinks \\\n0 Detection \n1 Detection \n2 Detection \n3 Detection \n4 Detection \n\n ProductName ProductComponentName AlertLink \\\n0 Azure Active Directory Identity Protection \n1 Azure Active Directory Identity Protection \n2 Azure Active Directory Identity Protection \n3 Microsoft Cloud App Security \n4 Microsoft Cloud App Security \n\n Status CompromisedEntity Tactics Type \n0 New Exploitation SecurityAlert \n1 New Exploitation SecurityAlert \n2 New Exploitation SecurityAlert \n3 New Exfiltration SecurityAlert \n4 New Execution SecurityAlert " }, "metadata": {} } ], "execution_count": 8, "metadata": {} }, { "cell_type": "markdown", "source": [ "# 4. Configure and test external data providers (VirusTotal and Maxmind GeoLite2)\n", "\n", "
\n", "Note: \n", "This section is optional although you are likely to need one or more Threat Intel providers.
\n", "You can also choose to use AlienVault OTX, IBM XForce or the MS Sentinel TI table (if\n", "you have configured TI import into MS Sentinel) in place of VirusTotal.
\n", "Follow the same procedures for the TI provider(s) of your choice.\n", "
\n", "\n", "Many Microsoft Sentinel notebooks make use of enrichment services such as Threat Intelligence and IP geo-location. We are going to set up two providers for these in this section.\n", "\n", "Since both providers have secret keys associated with their accounts we will also show you how to specify an Azure Key Vault to securely store these settings. This is optional - you can choose to store the keys in your msticpyconfig.yaml.\n", "\n", "\n", "## 4.1 (Optional) Configure Azure Key Vault to store secrets\n", "\n", "To store secrets in Azure Key Vault you need to have access to a Key Vault where you have permissions to read and write secrets.\n", "\n", "You can read more about this\n", "in the MSTICPY docs
\n", "If you want to skip this step, you can sign up for free accounts with both VirusTotal and MaxMind, until you can take the time to\n", "set up Key Vault storage.\n", "

\n", "\n", "You will need the following information about the Key Vault:\n", "- Azure Tenant ID (this is usually the same as you Microsoft Sentinel tenant)\n", "- Subscription ID that the KeyVault belongs to\n", "- Vault Name\n", "The ResourceGroup and AzureRegion are needed if you want to create a Key Vault using MSTICPy but are optional if the Vault has already been created.\n", "\n", "## Instructions\n", "1. Enter the **TenantId** and **Subscription**\n", "2. Enter the **Vault Name** - note: this is simple name, not the full URI of the Vaul.\n", "3. Click **Update**\n", "4. Click **Save Settings**\n" ], "metadata": {} }, { "cell_type": "code", "source": [ "display(mpedit)\n", "mpedit.set_tab(\"Key Vault\")" ], "outputs": [ { "output_type": "display_data", "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "b3a706d69d3f4812ab95091a943a81d1", "version_major": 2, "version_minor": 0 }, "text/plain": "VBox(children=(Tab(children=(VBox(children=(Label(value='Microsoft Sentinel workspace settings'), HBox(childre…" }, "metadata": {} } ], "execution_count": 15, "metadata": {} }, { "cell_type": "markdown", "source": [ "\n", "## 4.2 Configure and test Virus Total\n", "We are going to use [VirusTotal](https://www.virustotal.com) (VT) as an example of a popular threat intelligence source.\n", "To use VirusTotal threat intel lookups you will need a VirusTotal account and API key.\n", "\n", "You can sign up for a free account at the\n", "[VirusTotal getting started page](https://developers.virustotal.com/v3.0/reference#getting-started) website.\n", "\n", "If you are already a VirusTotal user, you can, of course, use your existing key.\n", "\n", "

\n", "Warning If you are using a VT enterprise key we do not recommend storing this\n", "in the msticpyconfig.yaml file.
\n", "MSTICPy supports storage of secrets in\n", "Azure Key Vault if you configured this in the previous step.\n", "\n", "\n", "As well as VirusTotal, we also support a range\n", "of other threat intelligence providers. You can read more about that here:\n", "[MSTICPy TI Providers](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html)\n", "\n", "### Instructions\n", "\n", "To add the VirusTotal details, run the following cell.\n", "\n", "1. Select \"VirusTotal\" from the **Add prov** drop down\n", "2. Click the **Add** button\n", "3. In the left-side Details panel select **Text** as the Storage option.\n", "4. Paste the API key in the **Value** text box.\n", "5. Click the **Update** button to confirm your changes.\n", "\n", "Your changes are not yet saved to your configuration file. To\n", "do this, click on the **Save Settings** button at the bottom of the dialog.\n", "\n", "If you are unclear about what anything in the configuration editor means, use the **Help** drop-down. This\n", "has instructions and links to more detailed documentation.\n" ], "metadata": {} }, { "cell_type": "code", "source": [ "mpe = msticpy.MpConfigEdit()\n", "mpe\n" ], "outputs": [ { "output_type": "display_data", "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "d712c52f75fe4883a19a11f563b1115f", "version_major": 2, "version_minor": 0 }, "text/plain": "Label(value='Loading. Please wait.')" }, "metadata": {} }, { "output_type": "display_data", "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "1782fe2ae4624828bddc3efa388c0cfc", "version_major": 2, "version_minor": 0 }, "text/plain": "VBox(children=(Tab(children=(VBox(children=(Label(value='Microsoft Sentinel workspace settings'), HBox(childre…" }, "metadata": {} } ], "execution_count": 10, "metadata": {} }, { "cell_type": "code", "source": [ "display(mpedit)\n", "mpedit.set_tab(\"TI Providers\")\n" ], "outputs": [ { "output_type": "display_data", "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "b3a706d69d3f4812ab95091a943a81d1", "version_major": 2, "version_minor": 0 }, "text/plain": "VBox(children=(Tab(children=(VBox(children=(Label(value='Microsoft Sentinel workspace settings'), HBox(childre…" }, "metadata": {} } ], "execution_count": 11, "metadata": {} }, { "cell_type": "markdown", "source": [ "Our notebooks commonly use IP geo-location information. \n", "In order to enable this we are going to set up [MaxMind GeoLite2](https://www.maxmind.com)\n", "to provide geolocation lookup services for IP addresses.\n", "\n", "GeoLite2 uses a downloaded database which requires an account key to download.\n", "You can sign up for a free account and a license key at \n", "[The Maxmind signup page - https://www.maxmind.com/en/geolite2/signup](https://www.maxmind.com/en/geolite2/signup).\n", "
\n", "\n", "

\n", " Using IPStack as an alernative to GeoLite2...\n", "

\n", " For more details see the\n", " \n", " MSTICPy GeoIP Providers documentation\n", "

\n", "
\n", "
\n", "\n", "Once, you have an account, run the following cell to add the Maxmind GeopIP Lite details to your configuration.\n", "\n", "### Instructions\n", "\n", "The procedure is similar to the one we used for VirusTotal:\n", "\n", "1. Select the \"GeoIPLite\" provider from the **Add prov** drop-down\n", "2. Click **Add**\n", "3. Select **Text** Storage and paste the license (API/Auth) key into the text box\n", "4. Click **Update**\n", "5. Click **Save Settings** to write your settings to your configuration.\n" ], "metadata": {} }, { "cell_type": "code", "source": [ "display(mpedit)\n", "mpedit.set_tab(\"GeoIP Providers\")\n" ], "outputs": [ { "output_type": "display_data", "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "b3a706d69d3f4812ab95091a943a81d1", "version_major": 2, "version_minor": 0 }, "text/plain": "VBox(children=(Tab(children=(VBox(children=(Label(value='Microsoft Sentinel workspace settings'), HBox(childre…" }, "metadata": {} } ], "execution_count": 12, "metadata": {} }, { "cell_type": "markdown", "source": [ "---\n", "\n", "## 4.1. Testing VirusTotal Lookup\n", "\n", "Threat intelligence and IP location are two common enrichments that you might apply to queried data.\n", "\n", "Let's test the VirusTotal provider with a known bad IP Address.\n", "\n", "
\n", " Learn more...\n", "

\n", "

\n", " \n", "
\n", "
" ], "metadata": {} }, { "cell_type": "code", "source": [ "# Refresh any config items that saved\n", "# to the msticpyconfig in the previous steps.\n", "msticpy.settings.refresh_config()\n", "\n", "# Create our TI provider\n", "ti = TILookup()\n", "\n", "# Lookup an IP Address\n", "ti_resp = ti.lookup_ioc(\"85.214.149.236\", providers=[\"VirusTotal\"])\n", "\n", "ti_df = ti.result_to_df(ti_resp)\n", "ti.browse_results(ti_df, severities=\"all\")" ], "outputs": [ { "output_type": "stream", "name": "stdout", "text": "Using Open PageRank. See https://www.domcop.com/openpagerank/what-is-openpagerank\n" }, { "output_type": "display_data", "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "0c32d03deb0d42d59da17a89f238c544", "version_major": 2, "version_minor": 0 }, "text/plain": "VBox(children=(Text(value='', description='Filter:', style=DescriptionStyle(description_width='initial')), Sel…" }, "metadata": {} }, { "output_type": "display_data", "data": { "text/html": "
", "text/plain": "" }, "metadata": {} }, { "output_type": "display_data", "data": { "text/html": "

85.214.149.236

Type: 'ipv4', Provider: VirusTotal, severity: high

Details

\n\n
VirusTotal
verbose_msgIP address in dataset
response_code1
positives69
detected_urls['http://85.214.149.236/sugarcrm/themes/default/images/', 'http://85.214.149.236:443/sugarcrm/themes/default/images/stock.jpg', 'http://85.214.149.236:443/sugarcrm/themes/default/images/', 'http://dl1.chimaera.cc/', 'http://85.214.149.236/', 'http://85.214.149.236/sugarcrm/themes/default/images/sugarlogic/.../tntb/containerpwn', 'http://85.214.149.236/sugarcrm/themes/default/images/SugarLogic/.../TNTb/ContainerPwn']
detected_downloaded_samples[]
detected_communicating_samples['c8895af7e57cf693d1dde9b3a361d03f14be0cdb2ee9c121496ea0315f06636a']

Reference:

https://www.virustotal.com/vtapi/v2/ip-address/report

Raw Results

\n
\n Raw results from provider...\n 
{'as_owner': 'Strato AG',
 'asn': 6724,
 'country': 'DE',
 'detected_communicating_samples': [{'date': '2021-11-22 10:45:43',
                                     'positives': 26,
                                     'sha256': 'c8895af7e57cf693d1dde9b3a361d03f14be0cdb2ee9c121496ea0315f06636a',
                                     'total': 72},
                                    {'date': '2021-09-24 21:40:34',
                                     'positives': 32,
                                     'sha256': '36bf7b2ab7968880ccc696927c03167b6056e73043fd97a33d2468383a5bafce',
                                     'total': 73},
                                    {'date': '2021-09-10 17:06:30',
                                     'positives': 2,
                                     'sha256': '132083d595f67afb43740f78b802015944c8e440bc5d42f54fc26522cba8e71b',
                                     'total': 73},
                                    {'date': '2021-09-10 17:05:47',
                                     'positives': 2,
                                     'sha256': '1b1d8a2cbb4b31bb9ee3ef94b788e882f40a9689ff90b17cb2c05bef50d5bdc8',
                                     'total': 73},
                                    {'date': '2021-08-22 17:26:46',
                                     'positives': 2,
                                     'sha256': 'fa9b38a2bd1acfd6b1b24af27cb82ea5620502d7e9cb8a913dceb897f2bcf87c',
                                     'total': 73},
                                    {'date': '2021-08-18 08:51:58',
                                     'positives': 15,
                                     'sha256': 'a4000315471cf197c0552aeec0e7afbe0a935b86ff9afe5b1443812d3f7185fa',
                                     'total': 75},
                                    {'date': '2021-08-15 11:10:16',
                                     'positives': 1,
                                     'sha256': '7bb1bd97dc93f0acf22eff6a5cbd9be685d18c8dbc982a24219928159c916c69',
                                     'total': 73},
                                    {'date': '2021-08-15 11:10:45',
                                     'positives': 1,
                                     'sha256': '451a4cbb6b931d8bb8392f08e7c9ec517b1b1ef06f42e1c8105e4feaafd6b157',
                                     'total': 73},
                                    {'date': '2021-07-29 04:49:18',
                                     'positives': 1,
                                     'sha256': '3cc54142b5f88d03fb0552a655e32e94f366c9e3bb387404c6f381cfea506867',
                                     'total': 74},
                                    {'date': '2021-07-26 16:18:47',
                                     'positives': 1,
                                     'sha256': '6c8a2ba339141b93c67f9d79d86a469da75bfbc69f128a6ed702a6e3925d5a29',
                                     'total': 74},
                                    {'date': '2021-06-11 01:23:22',
                                     'positives': 13,
                                     'sha256': 'ab12b5d03f8467a1089d3d40ef9c4a54fb16ee61bb68714040e9edf96b5e763f',
                                     'total': 74},
                                    {'date': '2021-06-10 07:31:53',
                                     'positives': 30,
                                     'sha256': '1aaf7bc48ff75e870db4fe6ec0b3ed9d99876d7e2fb3d5c4613cca92bbb95e1b',
                                     'total': 75},
                                    {'date': '2021-05-17 21:40:23',
                                     'positives': 13,
                                     'sha256': '39ac019520a278e350065d12ebc0c24201584390724f3d8e0dc828664fee6cae',
                                     'total': 74},
                                    {'date': '2021-05-12 12:46:23',
                                     'positives': 6,
                                     'sha256': 'b60ffcc7153650d6a232b1cb249924b0c6384c27681860eb13b12f4705bc0a05',
                                     'total': 75},
                                    {'date': '2021-05-11 08:32:51',
                                     'positives': 14,
                                     'sha256': '1ad0104478301e73e3f49cdeb10f8c1a1d54bccf9248e34ff81352598f112e6b',
                                     'total': 75},
                                    {'date': '2021-04-21 10:08:11',
                                     'positives': 16,
                                     'sha256': '7b6f7c48256a8df2041e8726c3490ccb6987e1a76fee947e148ea68eee036889',
                                     'total': 76},
                                    {'date': '2021-03-31 15:34:40',
                                     'positives': 20,
                                     'sha256': 'ae3e4a1c8a2b661265e6c8c756e3ba472dc7177cae79fe1861ab0c2d1af5167a',
                                     'total': 75},
                                    {'date': '2021-03-27 04:35:12',
                                     'positives': 22,
                                     'sha256': '3b280a4017ef2c2aef4b3ed8bb47516b816166998462899935afb39b533890ad',
                                     'total': 75},
                                    {'date': '2020-08-18 19:53:07',
                                     'positives': 3,
                                     'sha256': '0742efecbd7af343213a50cc5fd5cd2f8475613cfe6fb51f4296a7ec4533940d',
                                     'total': 74}],
 'detected_downloaded_samples': [{'date': '2021-11-13 17:03:39',
                                  'positives': 18,
                                  'sha256': '9245bb5d788677b0d5052eabf3897fa651e86110c6c32421821749eac0390e48',
                                  'total': 72},
                                 {'date': '2021-11-06 07:09:36',
                                  'positives': 28,
                                  'sha256': '33c8591edd61c6e968e727683a63fba0352b5b6b59a0b3005628c38848dd7dd3',
                                  'total': 74},
                                 {'date': '2021-10-22 08:14:33',
                                  'positives': 25,
                                  'sha256': 'f1a788466de258751a50e78cc97212c379e96b48e0ea22d62471083abd1346ef',
                                  'total': 74},
                                 {'date': '2021-09-09 10:17:59',
                                  'positives': 3,
                                  'sha256': '7bb1bd97dc93f0acf22eff6a5cbd9be685d18c8dbc982a24219928159c916c69',
                                  'total': 72},
                                 {'date': '2021-08-01 07:08:07',
                                  'positives': 25,
                                  'sha256': '4d7079a55d6d56973448fe0097724da16d72e1ac9db3bfce251eb39535fdbe0b',
                                  'total': 74},
                                 {'date': '2021-09-08 14:04:43',
                                  'positives': 1,
                                  'sha256': '08ed971ffbd71fc91f970c763313a1e7e37787346c2515a03e6dd9bab1a3f2a8',
                                  'total': 72},
                                 {'date': '2021-09-08 07:06:15',
                                  'positives': 23,
                                  'sha256': '2075c0835573b0004908da84e99f76960a13ea865b9effa847e3f61d43eff867',
                                  'total': 74},
                                 {'date': '2021-06-29 11:54:16',
                                  'positives': 26,
                                  'sha256': '75a733d99d72d1d0d6ca99ec852d97ae8c515ed136e12195e96adf6df7bbad41',
                                  'total': 75},
                                 {'date': '2021-07-12 18:36:35',
                                  'positives': 18,
                                  'sha256': 'be225e89211a3667e758a133bf75270daf1bb000672b5b4ba7b6337166e1c6f7',
                                  'total': 75},
                                 {'date': '2021-08-01 07:30:29',
                                  'positives': 34,
                                  'sha256': 'e15550481e89dbd154b875ce50cc5af4b49f9ff7b837d9ac5b5594e5d63966a3',
                                  'total': 69},
                                 {'date': '2021-06-16 21:05:31',
                                  'positives': 16,
                                  'sha256': 'e9c16ae54a5ca74c9e14adf940417831d560f0c1f542d6c25cb8cb76242bdedb',
                                  'total': 74},
                                 {'date': '2021-07-08 05:25:50',
                                  'positives': 35,
                                  'sha256': '0e574fd30e806fe4298b3cbccb8d1089454f42f52892f87554325cb352646049',
                                  'total': 75},
                                 {'date': '2021-07-08 08:53:31',
                                  'positives': 36,
                                  'sha256': '252bf8c685289759b90c1de6f9db345c2cfe62e6f8aad9a7f44dfb3c8508487a',
                                  'total': 74},
                                 {'date': '2021-07-08 10:29:28',
                                  'positives': 36,
                                  'sha256': 'a506c6cf25de202e6b2bf60fe0236911a6ff8aa33f12a78edad9165ab0851caf',
                                  'total': 75},
                                 {'date': '2021-07-08 08:53:29',
                                  'positives': 38,
                                  'sha256': '139f393594aabb20543543bd7d3192422b886f58e04a910637b41f14d0cad375',
                                  'total': 75},
                                 {'date': '2021-03-02 07:13:18',
                                  'positives': 33,
                                  'sha256': 'feb0a0f5ffba9d7b7d6878a8890a6d67d3f8ef6106e4e88719a63c3351e46a06',
                                  'total': 76},
                                 {'date': '2021-02-08 02:39:20',
                                  'positives': 18,
                                  'sha256': '230e2a06df2cd7574ee15cb13714d77182f28d50f83a6ed58af39f1966177769',
                                  'total': 76},
                                 {'date': '2020-10-31 16:15:20',
                                  'positives': 30,
                                  'sha256': '36bf7b2ab7968880ccc696927c03167b6056e73043fd97a33d2468383a5bafce',
                                  'total': 76},
                                 {'date': '2020-10-19 16:08:06',
                                  'positives': 28,
                                  'sha256': '1aaf7bc48ff75e870db4fe6ec0b3ed9d99876d7e2fb3d5c4613cca92bbb95e1b',
                                  'total': 75},
                                 {'date': '2020-09-09 11:54:11',
                                  'positives': 24,
                                  'sha256': '9750b3be953bd31322dd173ca18f29e5997029b28b24fbeb5fec7ebb1974cb09',
                                  'total': 73},
                                 {'date': '2020-09-06 07:41:39',
                                  'positives': 23,
                                  'sha256': 'c0ab7d1caabdd090b2399cd1193d2cc2334218d3f3f0d3164b61b6014fd308e9',
                                  'total': 73},
                                 {'date': '2020-09-09 11:30:10',
                                  'positives': 1,
                                  'sha256': '767fc7ae032403bce2dbcefa525cf1a9fd02bbb185e45aa88d9bc28a5f22a2b6',
                                  'total': 73},
                                 {'date': '2020-07-22 02:02:29',
                                  'positives': 26,
                                  'sha256': '132df864f6750d29bf9f762b298f377c13b899aa8d07c0a6bda58adcffd0d6f7',
                                  'total': 76},
                                 {'date': '2020-08-20 06:57:04',
                                  'positives': 30,
                                  'sha256': '2c40b76408d59f906f60db97ea36503bfc59aed22a154f5d564d8449c300594f',
                                  'total': 75}],
 'detected_referrer_samples': [{'date': '2020-09-09 11:30:10',
                                'positives': 1,
                                'sha256': '767fc7ae032403bce2dbcefa525cf1a9fd02bbb185e45aa88d9bc28a5f22a2b6',
                                'total': 73}],
 'detected_urls': [{'positives': 8,
                    'scan_date': '2021-12-22 08:31:13',
                    'total': 93,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/'},
                   {'positives': 8,
                    'scan_date': '2021-12-21 23:31:02',
                    'total': 93,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/stock.jpg'},
                   {'positives': 6,
                    'scan_date': '2021-12-20 16:27:00',
                    'total': 93,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/'},
                   {'positives': 4,
                    'scan_date': '2021-12-20 09:56:05',
                    'total': 93,
                    'url': 'http://dl1.chimaera.cc/'},
                   {'positives': 4,
                    'scan_date': '2021-12-17 06:34:35',
                    'total': 93,
                    'url': 'http://85.214.149.236/'},
                   {'positives': 7,
                    'scan_date': '2021-12-10 03:20:28',
                    'total': 93,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/sugarlogic/.../tntb/containerpwn'},
                   {'positives': 6,
                    'scan_date': '2021-12-09 12:07:40',
                    'total': 93,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/SugarLogic/.../TNTb/ContainerPwn'},
                   {'positives': 6,
                    'scan_date': '2021-11-22 08:57:10',
                    'total': 93,
                    'url': 'http://85.214.149.236:443/sugarcrm/'},
                   {'positives': 7,
                    'scan_date': '2021-11-15 18:18:07',
                    'total': 93,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../masscan/x86_64'},
                   {'positives': 7,
                    'scan_date': '2021-11-10 02:32:01',
                    'total': 93,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../jq/%D1%8786_64'},
                   {'positives': 7,
                    'scan_date': '2021-11-09 21:58:51',
                    'total': 93,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../N/NVIDIA-Linux-x86_64-470.57.02.run'},
                   {'positives': 10,
                    'scan_date': '2021-11-08 04:50:02',
                    'total': 93,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../TNTb/irc.chimaera.cc'},
                   {'positives': 7,
                    'scan_date': '2021-11-06 10:52:20',
                    'total': 93,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/SugarLogic/.../TNTb/irc.chimaera.cc'},
                   {'positives': 4,
                    'scan_date': '2021-11-02 15:58:50',
                    'total': 92,
                    'url': 'tcp://85.214.149.236:443/'},
                   {'positives': 7,
                    'scan_date': '2021-11-02 15:51:08',
                    'total': 92,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/SugarLogic/win/xmrig-6.13.1-msvc-win64.zip'},
                   {'positives': 7,
                    'scan_date': '2021-10-22 22:31:44',
                    'total': 91,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../pnscan/x86_64'},
                   {'positives': 6,
                    'scan_date': '2021-10-22 16:30:39',
                    'total': 91,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../xmr/kuben3/aarch64.tar.gz'},
                   {'positives': 6,
                    'scan_date': '2021-10-12 01:45:21',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm'},
                   {'positives': 6,
                    'scan_date': '2021-10-06 17:06:46',
                    'total': 90,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/sugarlogic/win/xmrig-6.13.1-msvc-win64.zip'},
                   {'positives': 10,
                    'scan_date': '2021-10-04 13:10:05',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes'},
                   {'positives': 9,
                    'scan_date': '2021-10-04 12:48:23',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/'},
                   {'positives': 8,
                    'scan_date': '2021-09-28 16:06:57',
                    'total': 89,
                    'url': 'http://dl1.chimaera.cc/sugarcrm/themes/default/images/sugarlogic/.../pnscan/x86_64'},
                   {'positives': 9,
                    'scan_date': '2021-09-28 06:27:21',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../docker/x86_64.tgz'},
                   {'positives': 11,
                    'scan_date': '2021-09-20 06:45:49',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/sugarlogic/.../tntb/x86_64'},
                   {'positives': 6,
                    'scan_date': '2021-09-18 15:59:59',
                    'total': 89,
                    'url': 'https://dl1.chimaera.cc/'},
                   {'positives': 7,
                    'scan_date': '2021-09-18 03:57:09',
                    'total': 89,
                    'url': 'http://dl1.chimaera.cc:443/sugarcrm/themes/default/images/SugarLogic/.../pnscan/x86_64'},
                   {'positives': 10,
                    'scan_date': '2021-09-17 22:17:09',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../curl/x86_64'},
                   {'positives': 9,
                    'scan_date': '2021-09-17 13:18:04',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../TNTb/x86_64'},
                   {'positives': 9,
                    'scan_date': '2021-09-15 15:52:06',
                    'total': 90,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/sugarlogic/win/init.bat'},
                   {'positives': 9,
                    'scan_date': '2021-09-10 11:37:03',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/win/init.bat'},
                   {'positives': 10,
                    'scan_date': '2021-09-09 13:30:06',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../TNTb'},
                   {'positives': 9,
                    'scan_date': '2021-09-09 13:11:28',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../TNTb/'},
                   {'positives': 10,
                    'scan_date': '2021-09-09 08:52:51',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/win'},
                   {'positives': 9,
                    'scan_date': '2021-09-07 15:30:52',
                    'total': 90,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/sugarlogic/.../xmr/sx/xmrig.so'},
                   {'positives': 10,
                    'scan_date': '2021-09-03 15:10:07',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../xmr/kuben3'},
                   {'positives': 9,
                    'scan_date': '2021-09-03 14:47:44',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../xmr/kuben3/'},
                   {'positives': 9,
                    'scan_date': '2021-09-03 14:45:43',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../xmr/sx/xmrig-6.13.1-linux-static-x64.tar.gz'},
                   {'positives': 9,
                    'scan_date': '2021-09-03 14:38:11',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../xmr/sx/3.sh'},
                   {'positives': 12,
                    'scan_date': '2021-08-27 19:43:46',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/bioset.jpg'},
                   {'positives': 9,
                    'scan_date': '2021-08-26 15:28:18',
                    'total': 90,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/sugarlogic/.../tntb/irc.chimaera.cc'},
                   {'positives': 9,
                    'scan_date': '2021-08-25 15:39:16',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../xmr/sx/xmrig.tar.gz'},
                   {'positives': 9,
                    'scan_date': '2021-08-23 11:38:06',
                    'total': 90,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/SugarLogic/.../xmr/x86_64'},
                   {'positives': 10,
                    'scan_date': '2021-08-23 02:56:48',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../TNTb/ContainerPwn'},
                   {'positives': 9,
                    'scan_date': '2021-08-22 02:17:32',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../xmr/x86_64.tar.gz'},
                   {'positives': 4,
                    'scan_date': '2021-08-21 23:59:21',
                    'total': 89,
                    'url': 'http://dl1.chimaera.cc:443/sugarcrm/themes/default/images/SugarLogic/.../masscan/x86_64'},
                   {'positives': 9,
                    'scan_date': '2021-08-18 15:35:11',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/sugarlogic/.../tntb/containerpwn'},
                   {'positives': 3,
                    'scan_date': '2021-08-13 15:27:24',
                    'total': 89,
                    'url': 'http://dl1.chimaera.cc:443/sugarcrm/themes/default/images/SugarLogic/...'},
                   {'positives': 13,
                    'scan_date': '2021-08-13 14:53:47',
                    'total': 89,
                    'url': 'http://dockerupdate.anondns.net/sugarcrm/themes/default/images/nk.jpg'},
                   {'positives': 12,
                    'scan_date': '2021-08-01 03:21:20',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/tshd.jpg'},
                   {'positives': 9,
                    'scan_date': '2021-07-31 20:54:57',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/kube.jpg'},
                   {'positives': 10,
                    'scan_date': '2021-07-30 09:13:40',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/default.jpg'},
                   {'positives': 9,
                    'scan_date': '2021-07-29 05:22:56',
                    'total': 90,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/SugarLogic/.../zgrab/x86_64'},
                   {'positives': 9,
                    'scan_date': '2021-06-28 02:32:40',
                    'total': 89,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/mod.jpg'},
                   {'positives': 12,
                    'scan_date': '2021-06-23 12:00:19',
                    'total': 88,
                    'url': 'http://dockerupdate.anondns.net/'},
                   {'positives': 12,
                    'scan_date': '2021-06-21 01:57:07',
                    'total': 88,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images/'},
                   {'positives': 7,
                    'scan_date': '2021-06-16 08:08:57',
                    'total': 89,
                    'url': 'http://85.214.149.236:443/'},
                   {'positives': 8,
                    'scan_date': '2021-06-09 03:40:07',
                    'total': 89,
                    'url': 'https://85.214.149.236/sugarcrm/themes/default/images'},
                   {'positives': 7,
                    'scan_date': '2021-06-09 03:18:37',
                    'total': 89,
                    'url': 'https://85.214.149.236/sugarcrm/themes/default/images/'},
                   {'positives': 8,
                    'scan_date': '2021-06-08 15:50:06',
                    'total': 89,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images'},
                   {'positives': 6,
                    'scan_date': '2021-04-21 00:07:34',
                    'total': 87,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/stock.jpg'},
                   {'positives': 5,
                    'scan_date': '2021-04-01 13:42:58',
                    'total': 85,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/mod.jpg'},
                   {'positives': 9,
                    'scan_date': '2021-03-19 18:12:09',
                    'total': 85,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images/Carray.jpg'},
                   {'positives': 6,
                    'scan_date': '2021-01-12 10:34:27',
                    'total': 83,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/kube.jpg'},
                   {'positives': 10,
                    'scan_date': '2020-12-28 02:17:00',
                    'total': 83,
                    'url': 'http://dockerupdate.anondns.net/sugarcrm/themes/default/images/'},
                   {'positives': 6,
                    'scan_date': '2020-12-19 10:34:37',
                    'total': 83,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images/default.jpg'},
                   {'positives': 6,
                    'scan_date': '2020-11-12 16:50:51',
                    'total': 81,
                    'url': 'http://85.214.149.236/sugarcrm/themes'},
                   {'positives': 14,
                    'scan_date': '2020-11-10 11:01:42',
                    'total': 81,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images/mos.jpg'},
                   {'positives': 14,
                    'scan_date': '2020-11-08 15:00:49',
                    'total': 81,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images/nk.jpg'},
                   {'positives': 6,
                    'scan_date': '2020-11-04 19:21:25',
                    'total': 81,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default'},
                   {'positives': 6,
                    'scan_date': '2020-10-29 00:55:07',
                    'total': 81,
                    'url': 'http://85.214.149.236/sugarcrm/themes/default/images'},
                   {'positives': 12,
                    'scan_date': '2020-09-28 03:26:34',
                    'total': 80,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm'},
                   {'positives': 9,
                    'scan_date': '2020-09-28 03:06:19',
                    'total': 80,
                    'url': 'http://85.214.149.236/sugarcrm/.../dns'},
                   {'positives': 11,
                    'scan_date': '2020-09-24 14:01:08',
                    'total': 80,
                    'url': 'http://dockerupdate.anondns.net/sugarcrm/themes/default/images'},
                   {'positives': 12,
                    'scan_date': '2020-09-21 17:20:19',
                    'total': 80,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images/carray.jpg'},
                   {'positives': 6,
                    'scan_date': '2020-09-20 16:04:57',
                    'total': 80,
                    'url': 'http://85.214.149.236/sugarcrm'},
                   {'positives': 9,
                    'scan_date': '2020-09-17 17:36:08',
                    'total': 80,
                    'url': 'http://85.214.149.236:443/sugarcrm/banner.php'},
                   {'positives': 11,
                    'scan_date': '2020-09-10 07:55:21',
                    'total': 80,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/'},
                   {'positives': 10,
                    'scan_date': '2020-09-09 12:06:14',
                    'total': 80,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images/Carray.jpg/'},
                   {'positives': 4,
                    'scan_date': '2020-09-09 12:05:12',
                    'total': 80,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/bioset.jpg/'},
                   {'positives': 11,
                    'scan_date': '2020-09-09 11:59:35',
                    'total': 80,
                    'url': 'http://dockerupdate.anondns.net/sugarcrm/themes/default/images/mos.jpg'},
                   {'positives': 5,
                    'scan_date': '2020-09-09 11:48:55',
                    'total': 80,
                    'url': 'http://85.214.149.236:443/sugarcrm/.../run'},
                   {'positives': 4,
                    'scan_date': '2020-09-09 11:44:28',
                    'total': 80,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/mod.js'},
                   {'positives': 6,
                    'scan_date': '2020-09-09 11:35:26',
                    'total': 80,
                    'url': 'http://85.214.149.236:443/sugarcrm/.../cron.sh'},
                   {'positives': 5,
                    'scan_date': '2020-09-05 03:44:35',
                    'total': 80,
                    'url': 'http://85.214.149.236/sugarcrm/...'},
                   {'positives': 8,
                    'scan_date': '2020-09-02 06:09:23',
                    'total': 80,
                    'url': 'https://dockerupdate.anondns.net/'},
                   {'positives': 6,
                    'scan_date': '2020-09-01 17:37:50',
                    'total': 79,
                    'url': 'http://85.214.149.236:443/sugarcrm/.../dns'},
                   {'positives': 1,
                    'scan_date': '2020-08-28 08:15:47',
                    'total': 78,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/mod.js\"'},
                   {'positives': 2,
                    'scan_date': '2020-08-27 13:22:06',
                    'total': 78,
                    'url': 'http://85.214.149.236:443/sugarcrm/themes/default/images/zgrab.jpg'},
                   {'positives': 7,
                    'scan_date': '2020-08-25 14:52:00',
                    'total': 79,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images/portjoe.jpg'},
                   {'positives': 7,
                    'scan_date': '2020-08-25 07:02:55',
                    'total': 79,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images'},
                   {'positives': 4,
                    'scan_date': '2020-08-24 07:34:44',
                    'total': 79,
                    'url': 'http://dockerupdate.anondns.net:443/sugarcrm/themes/default/images/01.jpg'}],
 'resolutions': [{'hostname': 'dl1.chimaera.cc',
                  'last_resolved': '2021-08-13 07:43:54'},
                 {'hostname': 'dockerupdate.anondns.net',
                  'last_resolved': '2020-08-14 18:56:08'},
                 {'hostname': 'h2381205.stratoserver.net',
                  'last_resolved': '2020-08-06 12:19:57'}],
 'response_code': 1,
 'undetected_communicating_samples': [{'date': '2021-11-17 10:12:13',
                                       'positives': 0,
                                       'sha256': 'a3b160e7c58fd879ce9eac6732adfef16fb554a1723ac86e2b31eb2b1d0fbef8',
                                       'total': 71},
                                      {'date': '2021-09-10 17:39:35',
                                       'positives': 0,
                                       'sha256': '48f92bdc4c039437ba77e6c6a74bb0d4b747aa94fb815223ea6d735d04fcb733',
                                       'total': 72},
                                      {'date': '2021-08-21 20:20:58',
                                       'positives': 0,
                                       'sha256': '0085bf33d4e4e051a15a1bd70636055d709aeef79025080afc7a8148ece55339',
                                       'total': 73},
                                      {'date': '2021-08-19 21:32:51',
                                       'positives': 0,
                                       'sha256': '0dab485f5eacbbaa62c2dd5385a67becf2c352f2ebedd2b5184ab4fba89d8f19',
                                       'total': 73},
                                      {'date': '2021-06-24 10:15:37',
                                       'positives': 0,
                                       'sha256': '7149b53e4a3f9de2a7d47190af64f8b609618ed09f8440a64175049a90336775',
                                       'total': 75},
                                      {'date': '2021-06-09 10:51:49',
                                       'positives': 0,
                                       'sha256': '45cc4f38340bf1d4bb0010114ccf03112d14dee7815aa797d20854605fdca2d2',
                                       'total': 74},
                                      {'date': '2021-06-12 19:00:20',
                                       'positives': 0,
                                       'sha256': '020531aef7e069ee6e384f2fe9c49db9d99292d559c72da95276c2788b17d386',
                                       'total': 74},
                                      {'date': '2020-12-10 15:39:02',
                                       'positives': 0,
                                       'sha256': 'd9c46904d5bb808f2f0c28e819a31703f5155c4df66c4c4669f5d9e81f25dc66',
                                       'total': 75},
                                      {'date': '2020-08-28 07:36:29',
                                       'positives': 0,
                                       'sha256': 'd333c3cfb8b9ad1da5ee50f96a55dfbe70196f05fd88b5f04e925e32305cfff8',
                                       'total': 73},
                                      {'date': '2020-08-28 07:40:32',
                                       'positives': 0,
                                       'sha256': '18c178fb224ec17718e5f70a92041e721d9e380e70063cc4bfe3f61d6feb72d9',
                                       'total': 73},
                                      {'date': '2020-08-28 07:35:10',
                                       'positives': 0,
                                       'sha256': 'a5d14bb053b03e81e58101516c782360fe64d6469852c6aa06fc47c08b30b127',
                                       'total': 73},
                                      {'date': '2020-08-26 22:30:40',
                                       'positives': 0,
                                       'sha256': 'ba16f24e6294e8da28782c1d8e00189950dd4cbbb061ef13d1a4d84305651768',
                                       'total': 73},
                                      {'date': '2020-08-26 14:29:14',
                                       'positives': 0,
                                       'sha256': '1861eee8333dadcfe0d0dc10461f5f82fada8e42db9aa9efba6f258182e9c546',
                                       'total': 73},
                                      {'date': '2020-08-24 07:12:27',
                                       'positives': 0,
                                       'sha256': 'b485e6ccc9cfeb9c2034cebfeaf1bb3b3db0ac9996e5260fc1e95ce852b757c4',
                                       'total': 73}],
 'undetected_downloaded_samples': [{'date': '2021-09-09 04:23:12',
                                    'positives': 0,
                                    'sha256': '162c6bdc92693559b937d7ec46d7e93441c1d414d2da823044fcfc57d8f546ce',
                                    'total': 73},
                                   {'date': '2020-09-09 11:44:35',
                                    'positives': 0,
                                    'sha256': 'e8812c8ff47b0542c7ee4d6bdff5bfbfd488a8e363d884074089c54b6ffc9789',
                                    'total': 73},
                                   {'date': '2020-07-16 04:03:02',
                                    'positives': 0,
                                    'sha256': '1474298ed7a5c63ca8098794cd743a276807cca0e678e046160718626bb038f3',
                                    'total': 76}],
 'undetected_referrer_samples': [{'date': '2021-12-15 17:04:22',
                                  'positives': 0,
                                  'sha256': 'ab971c2fd88fd5fdc413068143d0a3b0b0ab6b1b4b927a78001c1318299c555e',
                                  'total': 71},
                                 {'date': '2021-09-09 12:12:02',
                                  'positives': 0,
                                  'sha256': '63e44d333b4eb8e0585f8653a66c845f13c98787c9bd2b9c46b0563c8b5d4196',
                                  'total': 70}],
 'undetected_urls': [['http://h2381205.stratoserver.net/',
                      '011bcc2795245bb9fac15c54e0e189b0c6e2f24c42c57fec7cfc654a8bb95106',
                      0,
                      80,
                      '2020-11-02 13:02:39'],
                     ['http://85.214.149.236:443/sugarcrm/.../',
                      '9ffbd9455f6aa190b4270b0d8bfe2c863c6495b94b0f510169999135476e4ed4',
                      0,
                      79,
                      '2020-07-14 10:52:05']],
 'verbose_msg': 'IP address in dataset'}
\n
\n ", "text/plain": "" }, "metadata": {} } ], "execution_count": 13, "metadata": {} }, { "cell_type": "markdown", "source": [ "## 4.2 Test IP geolocation lookup with Maxmind GeoLite2\n", "\n", "
Note:\n", "You may see the GeoLite driver downloading its database the first time you run this.\n", "
\n", "
\n", "
\n", " Learn more about MSTICPy GeoIP providers...\n", "

\n", " MSTICPy GeoIP Providers\n", "

\n", "
\n", "
\n" ], "metadata": {} }, { "cell_type": "code", "source": [ "geo_ip = GeoLiteLookup()\n", "raw_res, ip_entity = geo_ip.lookup_ip(\"85.214.149.236\")\n", "display(ip_entity[0])" ], "outputs": [ { "output_type": "display_data", "data": { "text/html": "\nThis product includes GeoLite2 data created by MaxMind, available from\nhttps://www.maxmind.com.\n", "text/plain": "" }, "metadata": {} }, { "output_type": "display_data", "data": { "text/html": "

ipaddress

{ 'AdditionalData': {},
  'Address': '85.214.149.236',
  'Location': { 'AdditionalData': {},
                'CountryCode': 'DE',
                'CountryName': 'Germany',
                'Latitude': 51.2993,
                'Longitude': 9.491,
                'Type': 'geolocation'},
  'ThreatIntelligence': [],
  'Type': 'ipaddress'}", "text/plain": "IpAddress(Address=85.214.149.236, Location={ 'AdditionalData': {},\n 'CountryCode': 'DE',\n...)" }, "metadata": {} } ], "execution_count": 14, "metadata": {} }, { "cell_type": "markdown", "source": [ "---\n", "\n", "# 5. Conclusion and Next Steps\n", "\n", "In this notebook, we've gone through the basics of installing MSTICPy and setting up configuration.\n", "We also briefly introduced:\n", "\n", "- QueryProviders and querying data from Microsoft Sentinel\n", "- Threat Intelligence lookups using VirusTotal\n", "- Geo-location lookups using MaxMind GeoLite2\n", "\n", "## Next Steps\n", "We encourage you to run through the **A Tour of Cybersec notebook features** notebook\n", "to get a better feel for some more of the capabilities of notebooks and MSTICPy.
\n", "\n", "This notebook includes:\n", "\n", "- more examples of queries\n", "- visualizing your data\n", "- brief introduction to using panda to manipulate your data.\n", "\n", "Also try out some of the other Microsoft Sentinel notebooks:\n", "\n", "- Data Visualization:\n", " - A Tour of Cybersec notebook features\n", "- Investigation:\n", " - Guided Triage - Alerts\n", "- Hunting:\n", " - Entity Explorer - Account\n", " - Entity Explorer - Windows Host\n", " - Entity Explorer - Domain and URL\n", "- Simple Machine Learning:\n", " - Machine Learning in Notebooks Examples\n", "\n", "Also check out some of the other sample notebooks in the [Microsoft Sentinel Notebooks GitHub repository](https://github.com/Azure/Azure-Sentinel-Notebooks)\n" ], "metadata": {} }, { "cell_type": "markdown", "source": [ "---\n", "\n", "# 6. Futher resources\n", "\n", " - [Jupyter Notebooks: An Introduction](https://realpython.com/jupyter-notebook-introduction/)\n", " - [Threat Hunting in the cloud with Azure Notebooks](https://medium.com/@maarten.goet/threat-hunting-in-the-cloud-with-azure-notebooks-supercharge-your-hunting-skills-using-jupyter-8d69218e7ca0)\n", " - [MSTICPy documentation](https://msticpy.readthedocs.io/)\n", " - [Microsoft Sentinel Notebooks documentation](https://docs.microsoft.com/azure/sentinel/notebooks)\n", " - [The Infosec Jupyterbook](https://infosecjupyterbook.com/introduction.html)\n", " - [Linux Host Explorer Notebook walkthrough](https://techcommunity.microsoft.com/t5/azure-sentinel/explorer-notebook-series-the-linux-host-explorer/ba-p/1138273)\n", " - [Why use Jupyter for Security Investigations](https://techcommunity.microsoft.com/t5/azure-sentinel/why-use-jupyter-for-security-investigations/ba-p/475729)\n", " - [Security Investigtions with Microsoft Sentinel & Notebooks](https://techcommunity.microsoft.com/t5/azure-sentinel/security-investigation-with-azure-sentinel-and-jupyter-notebooks/ba-p/432921)\n", " - [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/user_guide/index.html)\n", " - [Bokeh Documentation](https://docs.bokeh.org/en/latest/)" ], "metadata": {} }, { "cell_type": "markdown", "source": [ "---\n", "\n", "# 7. FAQs\n", "\n", "The following links take you to short articles in the Azure-Sentinel-Notebooks Wiki\n", "that answer common questions.\n", "\n", "- [How can I download all Azure-Sentinel-Notebooks notebooks to my Azure ML workspace?](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/How-can-I-download-all-Azure-Sentinel-Notebooks-notebooks-to-my-Azure-ML-workspace%3F)\n", "\n", "- [Can I install MSTICPy by default on a new AML compute?](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/Can-I-install-MSTICPy-by-default-on-a-new-AML-compute%3F)\n", "\n", "- [I see error \"Runtime dependency of PyGObject is missing\" when I load a query provider](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/%22Runtime-dependency-of-PyGObject-is-missing%22-error)\n", "\n", "- [MSTICPy and other packages do not install properly when switching between the Python 3.6 or 3.8 Kernels](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/MSTICPy-and-other-packages-do-not-install-properly-when-switching-between-the-Python-3.6-or-3.8-Kernels)\n", "\n", "- [My user account/credentials do not get cached between notebook runs - using Azure CLI](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/Caching-credentials-with-Azure-CLI)\n", "\n", "See other FAQs here [Microsoft Sentinel Notebooks wiki](https://github.com/Azure/Azure-Sentinel-Notebooks/wiki/)" ], "metadata": {} } ], "metadata": { "interpreter": { "hash": "b736adfe05d9ae282eea4c01a733d58a0215ef3399d39339e6557e4c515b0f48" }, "kernelspec": { "name": "python38-azureml", "language": "python", "display_name": "Python 3.8 - AzureML" }, "language_info": { "name": "python", "version": "3.8.5", "mimetype": "text/x-python", "codemirror_mode": { "name": "ipython", "version": 3 }, "pygments_lexer": "ipython3", "nbconvert_exporter": "python", "file_extension": ".py" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } }, "microsoft": { "ms_spell_check": { "ms_spell_check_language": "en" } }, "kernel_info": { "name": "python38-azureml" }, "nteract": { "version": "nteract-front-end@1.0.0" } }, "nbformat": 4, "nbformat_minor": 4 }