{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Notebook Title\n", "
\n", "  Details...\n", "\n", " **Notebook Version:** 1.0
\n", " **Python Version:** Python 3.6 (including Python 3.6 - AzureML)
\n", " **Required Packages**: kqlmagic, msticpy, pandas, pandas_bokeh, numpy, matplotlib, networkx, seaborn, datetime, ipywidgets, ipython, dnspython, ipwhois, folium, maxminddb_geolite2
\n", " **Platforms Supported**:\n", " - Azure Notebooks Free Compute\n", " - Azure Notebooks DSVM\n", " - OS Independent\n", "\n", " **Data Sources Required**:\n", " - Log Analytics/Azure Sentinel - Syslog, Secuirty Alerts, Auditd, Azure Network Analytics.\n", " - (Optional) - AlienVault OTX (requires account and API key)\n", "
\n", "\n", "Notebook description...." ] }, { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

Table of Contents

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "### Notebook initialization\n", "The next cell:\n", "- Checks for the correct Python version\n", "- Checks versions and optionally installs required packages\n", "- Imports the required packages into the notebook\n", "- Sets a number of configuration options.\n", "\n", "This should complete without errors. If you encounter errors or warnings look at the following two notebooks:\n", "- [TroubleShootingNotebooks](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/TroubleShootingNotebooks.ipynb)\n", "- [ConfiguringNotebookEnvironment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)\n", "\n", "If you are running in the Azure Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:\n", "- [Run TroubleShootingNotebooks](./TroubleShootingNotebooks.ipynb)\n", "- [Run ConfiguringNotebookEnvironment](./ConfiguringNotebookEnvironment.ipynb)\n", "\n", "You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup. \n", "There are more details about this in the `ConfiguringNotebookEnvironment` notebook and in these documents:\n", "- [msticpy configuration](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)\n", "- [Threat intelligence provider configuration](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file)\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "ExecuteTime": { "end_time": "2020-05-16T02:00:38.505687Z", "start_time": "2020-05-16T02:00:31.727307Z" } }, "outputs": [], "source": [ "from pathlib import Path\n", "import os\n", "import sys\n", "import warnings\n", "from IPython.display import display, HTML, Markdown\n", "\n", "REQ_PYTHON_VER=(3, 6)\n", "REQ_MSTICPY_VER=(0, 5, 0)\n", "\n", "display(HTML(\"

Starting Notebook setup...

\"))\n", "if Path(\"./utils/nb_check.py\").is_file():\n", " from utils.nb_check import check_python_ver, check_mp_ver\n", "\n", " check_python_ver(min_py_ver=REQ_PYTHON_VER)\n", " try:\n", " check_mp_ver(min_msticpy_ver=REQ_MSTICPY_VER)\n", " except ImportError:\n", " !pip install --user --upgrade msticpy\n", " if \"msticpy\" in sys.modules:\n", " importlib.reload(msticpy)\n", " else:\n", " import msticpy\n", " check_mp_ver(MSTICPY_REQ_VERSION)\n", " \n", "from msticpy.nbtools import nbinit\n", "nbinit.init_notebook(\n", " namespace=globals(),\n", " extra_imports=[\"ipwhois, IPWhois\"]\n", ");\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[Contents](#toc)\n", "### Get WorkspaceId and Authenticate to Log Analytics \n", "<details>\n", " Details...\n", "If you are using user/device authentication, run the following cell. \n", "- Click the 'Copy code to clipboard and authenticate' button.\n", "- This will pop up an Azure Active Directory authentication dialog (in a new tab or browser window). The device code will have been copied to the clipboard. \n", "- Select the text box and paste (Ctrl-V/Cmd-V) the copied value. \n", "- You should then be redirected to a user authentication page where you should authenticate with a user account that has permission to query your Log Analytics workspace.\n", "\n", "Use the following syntax if you are authenticating using an Azure Active Directory AppId and Secret:\n", "```\n", "%kql loganalytics://tenant(aad_tenant).workspace(WORKSPACE_ID).clientid(client_id).clientsecret(client_secret)\n", "```\n", "instead of\n", "```\n", "%kql loganalytics://code().workspace(WORKSPACE_ID)\n", "```\n", "\n", "Note: you may occasionally see a JavaScript error displayed at the end of the authentication - you can safely ignore this.
\n", "On successful authentication you should see a ```popup schema``` button.\n", "To find your Workspace Id go to [Log Analytics](https://ms.portal.azure.com/#blade/HubsExtension/Resources/resourceType/Microsoft.OperationalInsights%2Fworkspaces). Look at the workspace properties to find the ID.\n", "</details>" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# List Workspaces available\n", "# WorkspaceConfig().list_workspaces()\n", "\n", "# To use a specific workspace create a WorkspaceConfig using the\n", "# workspace parameter\n", "# ws_config = WorkspaceConfig(workspace='MyWorkspace')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "ExecuteTime": { "end_time": "2020-05-16T02:01:11.022700Z", "start_time": "2020-05-16T02:00:49.394760Z" } }, "outputs": [], "source": [ "# Authentication\n", "ws_config = WorkspaceConfig()\n", "qry_prov = QueryProvider(data_environment=\"LogAnalytics\")\n", "qry_prov.connect(connection_str=ws_config.code_connect_str)\n", "table_index = qry_prov.schema_tables" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "ExecuteTime": { "end_time": "2020-05-16T02:03:12.112983Z", "start_time": "2020-05-16T02:03:12.055984Z" } }, "outputs": [], "source": [ "query_scope = nbwidgets.QueryTime(auto_display=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example query" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "ExecuteTime": { "end_time": "2020-05-16T02:03:25.227614Z", "start_time": "2020-05-16T02:03:21.291120Z" } }, "outputs": [], "source": [ "qry_prov.SecurityAlert.list_alerts(query_scope)" ] } ], "metadata": { "hide_input": false, "kernelspec": { "display_name": "Python 3.6", "language": "python", "name": "python36" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autoclose": false, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": false, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": true, "toc_window_display": true }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": {}, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 4 }