{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Sessionize, Model and Visualise Office Exchange Data\n",
"\n",
" Notebook Version: 1.0
\n",
" Python Version: Python 3.6 (including Python 3.6 - AzureML)
\n",
" Required Packages: msticpy, pandas, kqlmagic
\n",
"\n",
"Data Sources Required:\n",
"* Log Analytics - OfficeActivity\n",
"\n",
"Configuration Required:\n",
"\n",
"This Notebook presumes you have your Microsoft Sentinel Workspace settings configured in a config file. If you do not have this in place, please [read the docs](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html) and [use this notebook](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb) to test.\n",
"\n",
"\n",
"\n",
"## Description:\n",
"Various types of security logs can be broken up into sessions/sequences where each session can be thought of as an ordered sequence of events. It can be useful to model these sessions in order to understand what the usual activity is like so that we can highlight anomalous sequences of events.\n",
"\n",
"In this hunting notebook, we treat the Office Exchange PowerShell cmdlets (\"Set-Mailbox\", \"Set-MailboxFolderPermission\" etc) as \"events\" and then group the events into \"sessions\" on a per-user basis. We demonstrate the sessionizing, modelling and visualisation on the Office Exchange Admin logs, however the methods used in this notebook can be applied to other log types as well.\n",
"\n",
"A new subpackage called anomalous_sequence has been released to [msticpy](https://github.com/microsoft/msticpy/tree/master/msticpy/analysis/anomalous_sequence) recently. This library allows the user to sessionize, model and visualize their data via some high level functions. For more details on how to use this subpackage, please [read the docs](https://msticpy.readthedocs.io/en/latest/data_analysis/AnomalousSequence.html) and/or refer to this more [documentation heavy notebook](https://github.com/microsoft/msticpy/blob/master/docs/notebooks/AnomalousSequence.ipynb). The documentation for this subpackage also includes some suggested guidance on how this library can be applied to some other log types.\n",
"\n",
"\n",
"High level sections of the notebook:\n",
"* Sessionize your Office Exchange logs data using built-in KQL operators\n",
"* Use the anomalous_sequence subpackage of msticpy to model the sessions\n",
"* Use the anomalous_sequence subpackage of msticpy to visualize the scored sessions\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Table of Contents\n",
"* [Notebook Initialization](#init_notebook)\n",
" * [Imports](#imports)\n",
" * [Authenticate Log Analytics](#la_auth)\n",
"* [Create Sessions from your Office Exchange Data](#create_sessions)\n",
" * [What is a Session?](#create_sessions)\n",
" * [Sessionize using Kusto's Native Functionality](#use_la)\n",
" * [Convert sessions into an allowed format for the modelling](#clean_sessions)\n",
"* [Model the Sessions](#explain_model)\n",
" * [High Level function for modelling](#model_function)\n",
"* [Visualise the Modelled Sessions](#visualize_function)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notebook initialization \n",
"\n",
"The next cell:\n",
"\n",
"* Checks for the correct Python version\n",
"* Checks versions and optionally installs required packages\n",
"* Imports the required packages into the notebook\n",
"* Sets a number of configuration options\n",
"\n",
"This should complete without errors. If you encounter errors or warnings, please look at the following two notebooks:\n",
"\n",
"* [TroubleShootingNotebooks](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/TroubleShootingNotebooks.ipynb)\n",
"* [ConfiguringNotebookEnvironment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"gather": {
"logged": 1617756662780
},
"tags": []
},
"outputs": [],
"source": [
"from pathlib import Path\n",
"from IPython.display import display, HTML\n",
"\n",
"REQ_PYTHON_VER = \"3.6\"\n",
"REQ_MSTICPY_VER = \"1.0.0\"\n",
"\n",
"\n",
"display(HTML(\"