{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Querying Microsoft Defender Data\n", "MSTICPy versions >= 1.5.0\n", "\n", "### Description\n", "This Notebook provides details and examples of how to connect to and query data from the Microsoft Defender Advanced Hunting API.\n", "\n", "
Note: \n", "This notebook reflects a partially-updated component and still\n", "uses the \"MDATP\" abbreviation to refer to the Microsoft 365 Defender\n", "and Microsoft Defender for Endpoint data services.\n", "
\n", "\n", "### Installation\n", "\n", "```python\n", "%pip install --upgrade msticpy\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Authentication\n", "\n", "Authentication for the Microsoft Defender Advanced Hunting API is handled via an Azure AD application. Before you can authenticate you will need to register an application and provide it with the required permissions. MSTICpy supports Application Context authentication to the API.\n", "Detailed instructions on registering an application can be found here: \n", "- [Get access with an application context](https://docs.microsoft.com/microsoft-365/security/defender-endpoint/exposed-apis-create-app-webapp?view=o365-worldwide)\n", "- [Get access with a user context](https://docs.microsoft.com/microsoft-365/security/defender-endpoint/exposed-apis-create-app-nativeapp?view=o365-worldwide)\n", "\n", "Once created you will require the following details:\n", "* Application (client) ID\n", "* Directory (tenant) ID\n", "* Client secret\n", "\n", "These details can be found in the Azure Portal under Azure Active Directory > App Registrations.\n", "\n", "Once collected the easiest way to manage these details is via msticpyconfig.yaml - simply add them to the file in the following format:\n", "\n", "```yaml\n", "DataProviders:\n", " MicrosoftDefender:\n", " Args:\n", " ClientId: \"CLIENT ID\"\n", " ClientSecret:\n", " KeyVault: \n", " TenantId: \"TENANT ID\"\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can then initialize a data provider for Microsoft Defender and connect the provider.\n", "\n", "Note: you can also provide these values to the connect function.\n", "See [Microsoft Defender data provider](https://msticpy.readthedocs.io/en/latest/data_acquisition/DataProviders.html#microsoft-defender)\n", "\n", "Note: \n", "If you want to access the Microsoft Defender for Endpoint\n", "APIs rather than the M365 Defender API (the latter is a subset\n", "of the former), please use \"MDE\" as the parameter to QueryProvider.\n", "
" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Connected.\n" ] } ], "source": [ "from msticpy.data.data_providers import QueryProvider\n", "md_prov = QueryProvider(\"M365D\")\n", "\n", "md_prov.connect()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once connected the Microsoft Defender data connector functions in a similar manner to other data connectors. You can list queries:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['MDATP.file_path',\n", " 'MDATP.host_alerts',\n", " 'MDATP.host_connections',\n", " 'MDATP.ip_alerts',\n", " 'MDATP.ip_connections',\n", " 'MDATP.list_alerts',\n", " 'MDATP.list_connections',\n", " 'MDATP.list_filehash',\n", " 'MDATP.list_files',\n", " 'MDATP.list_host_processes',\n", " 'MDATP.process_cmd_line',\n", " 'MDATP.process_creations',\n", " 'MDATP.process_paths',\n", " 'MDATP.protocol_connections',\n", " 'MDATP.sha1_alerts',\n", " 'MDATP.url_alerts',\n", " 'MDATP.url_connections',\n", " 'MDATP.user_files',\n", " 'MDATP.user_logons',\n", " 'MDATP.user_network',\n", " 'MDATP.user_processes',\n", " 'MDATPHunting.accessibility_persistence',\n", " 'MDATPHunting.av_sites',\n", " 'MDATPHunting.b64_pe',\n", " 'MDATPHunting.brute_force',\n", " 'MDATPHunting.cve_2018_1000006l',\n", " 'MDATPHunting.cve_2018_1111',\n", " 'MDATPHunting.cve_2018_4878',\n", " 'MDATPHunting.doc_with_link',\n", " 'MDATPHunting.dropbox_link',\n", " 'MDATPHunting.email_link',\n", " 'MDATPHunting.email_smartscreen',\n", " 'MDATPHunting.malware_recycle',\n", " 'MDATPHunting.network_scans',\n", " 'MDATPHunting.powershell_downloads',\n", " 'MDATPHunting.service_account_powershell',\n", " 'MDATPHunting.smartscreen_ignored',\n", " 'MDATPHunting.smb_discovery',\n", " 'MDATPHunting.tor',\n", " 'MDATPHunting.uncommon_powershell',\n", " 'MDATPHunting.user_enumeration']" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "md_prov.list_queries()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Get details about avaliable queries:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Query: list_alerts\n", "Data source: MDE\n", "Retrieves list of alerts\n", "\n", "Parameters\n", "----------\n", "add_query_items: str (optional)\n", " Additional query clauses\n", "end: datetime\n", " Query end time\n", "start: datetime\n", " Query start time\n", "table: str (optional)\n", " Table name\n", " (default value is: DeviceAlertEvents)\n", "Query:\n", " {table} | where Timestamp >= datetime({start}) | where Timestamp <= datetime({end}) {add_query_items}\n" ] } ], "source": [ "md_prov.MDATP.list_alerts('?')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Execute queries with default parameters:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "\n", " | AlertId | \n", "Timestamp | \n", "DeviceId | \n", "DeviceName | \n", "Severity | \n", "Category | \n", "Title | \n", "FileName | \n", "SHA1 | \n", "RemoteUrl | \n", "RemoteIP | \n", "AttackTechniques | \n", "ReportId | \n", "Table | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "ed637828714417577685_889446803 | \n", "2022-03-14 16:03:11.918913300+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "High | \n", "UnwantedSoftware | \n", "Notepad Execution Detection | \n", "notepad.exe | \n", "5b80bbb07b1a84384e61fb3f9366cad97904ebea | \n", "\n", " | \n", " | \n", " | 25654 | \n", "Not supported | \n", "
1 | \n", "ed637828714417577685_889446803 | \n", "2022-03-14 16:03:11.918913300+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "High | \n", "UnwantedSoftware | \n", "Notepad Execution Detection | \n", "notepad.exe | \n", "5b80bbb07b1a84384e61fb3f9366cad97904ebea | \n", "\n", " | \n", " | \n", " | 25654 | \n", "DeviceProcessEvents | \n", "
2 | \n", "da637831027930681075_399960450 | \n", "2022-03-17 08:32:13.537246500+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "Low | \n", "Malware | \n", "'ATPMiniDump' hacktool was prevented | \n", "ATPMiniDump.exe | \n", "2659afd376e84ea8dab21d73412069677212abd6 | \n", "\n", " | \n", " | \n", " | 50216 | \n", "DeviceEvents | \n", "
3 | \n", "da637831027930681075_399960450 | \n", "2022-03-17 08:32:41.619019500+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "Low | \n", "Malware | \n", "'ATPMiniDump' hacktool was prevented | \n", "ATPMiniDump.exe | \n", "2659afd376e84ea8dab21d73412069677212abd6 | \n", "\n", " | \n", " | \n", " | 50232 | \n", "DeviceEvents | \n", "
4 | \n", "da637831027930681075_399960450 | \n", "2022-03-17 08:32:51.787218300+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "Low | \n", "Malware | \n", "'ATPMiniDump' hacktool was detected | \n", "ATPMiniDump.exe | \n", "2659afd376e84ea8dab21d73412069677212abd6 | \n", "\n", " | \n", " | \n", " | 50322 | \n", "DeviceEvents | \n", "
5 | \n", "da637831027930681075_399960450 | \n", "2022-03-17 08:32:51.795963100+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "Low | \n", "Malware | \n", "'ATPMiniDump' hacktool was detected | \n", "ATPMiniDump.exe | \n", "2659afd376e84ea8dab21d73412069677212abd6 | \n", "\n", " | \n", " | \n", " | 50351 | \n", "DeviceEvents | \n", "
6 | \n", "da637831029556671978_1988175326 | \n", "2022-03-17 08:33:14.289264600+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "Medium | \n", "Malware | \n", "An active 'ATPMiniDump' hacktool process was d... | \n", "ATPMiniDump.exe | \n", "2659afd376e84ea8dab21d73412069677212abd6 | \n", "\n", " | \n", " | \n", " | 50540 | \n", "DeviceEvents | \n", "
7 | \n", "da637831027930681075_399960450 | \n", "2022-03-17 08:33:14.289301300+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "Low | \n", "Malware | \n", "'ATPMiniDump' hacktool was detected | \n", "ATPMiniDump.exe | \n", "2659afd376e84ea8dab21d73412069677212abd6 | \n", "\n", " | \n", " | \n", " | 50541 | \n", "DeviceEvents | \n", "
8 | \n", "da637831027930681075_399960450 | \n", "2022-03-17 08:33:14.505995400+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "Low | \n", "Malware | \n", "'ATPMiniDump' hacktool was detected | \n", "ATPMiniDump.exe | \n", "2659afd376e84ea8dab21d73412069677212abd6 | \n", "\n", " | \n", " | \n", " | 50521 | \n", "DeviceEvents | \n", "
9 | \n", "da637831931160791947_311302490 | \n", "2022-03-18 09:36:44.691315900+00:00 | \n", "e09b459fcd3aa36f23084211c34bfca046848b65 | \n", "atevet06cl005.defenderatevet06.onmicrosoft.com | \n", "Medium | \n", "Execution | \n", "Suspicious PowerShell command line | \n", "powershell.exe | \n", "f43d9bb316e30ae1a3494ac5b0624f6bea1bf054 | \n", "\n", " | \n", " | [\"PowerShell (T1059.001)\"] | \n", "127261 | \n", "DeviceImageLoadEvents | \n", "
10 | \n", "da637831931152888965_11461089 | \n", "2022-03-18 09:36:44.691315900+00:00 | \n", "e09b459fcd3aa36f23084211c34bfca046848b65 | \n", "atevet06cl005.defenderatevet06.onmicrosoft.com | \n", "Medium | \n", "Execution | \n", "Suspicious PowerShell command line | \n", "powershell.exe | \n", "f43d9bb316e30ae1a3494ac5b0624f6bea1bf054 | \n", "\n", " | \n", " | [\"PowerShell (T1059.001)\"] | \n", "127261 | \n", "DeviceImageLoadEvents | \n", "
\n", " | Severity | \n", "count_ | \n", "
---|---|---|
0 | \n", "High | \n", "53 | \n", "
1 | \n", "Medium | \n", "11 | \n", "
2 | \n", "Low | \n", "19 | \n", "
3 | \n", "Informational | \n", "6 | \n", "
\n", " | AlertId | \n", "Timestamp | \n", "DeviceId | \n", "DeviceName | \n", "Severity | \n", "Category | \n", "Title | \n", "FileName | \n", "SHA1 | \n", "RemoteUrl | \n", "RemoteIP | \n", "AttackTechniques | \n", "ReportId | \n", "Table | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "da637807989907659907_-236366407 | \n", "2022-02-21 09:47:47.938238100+00:00 | \n", "efeb32222798ea6c2eb747514dd34e646cf751aa | \n", "win-6l54m49tpgn.woanware.co.uk | \n", "Low | \n", "Discovery | \n", "Suspicious System Service Discovery | \n", "powershell.exe | \n", "5b1a36a8b206fe9daece7e2cd702cdd4e10bb895 | \n", "\n", " | \n", " | [\"System Service Discovery (T1007)\",\"Windows M... | \n", "45923 | \n", "DeviceEvents | \n", "
1 | \n", "da637824441902905094_1268174690 | \n", "2022-03-09 17:34:15.638562100+00:00 | \n", "8a2d7be2ee62093058401b4d50c481fe9ec7aedb | \n", "attackervm.attack.vm | \n", "High | \n", "Persistence | \n", "A suspicious DLL was loaded by the ADFS service | \n", "version.dll | \n", "ebdcd4c1743adc9714a94a09eeb770c8b67747cc | \n", "\n", " | \n", " | [\"T1606\"] | \n", "718 | \n", "DeviceImageLoadEvents | \n", "
2 | \n", "da637807989904403259_2043617699 | \n", "2022-02-21 09:50:28.356919+00:00 | \n", "efeb32222798ea6c2eb747514dd34e646cf751aa | \n", "win-6l54m49tpgn.woanware.co.uk | \n", "Low | \n", "Discovery | \n", "Suspicious User Account Discovery | \n", "explorer.exe | \n", "dbf66924d79d246da08df40db1f20595385b6347 | \n", "\n", " | \n", " | [\"Remote System Discovery (T1018)\",\"PowerShell... | \n", "46011 | \n", "DeviceRegistryEvents | \n", "
3 | \n", "ed637810396041195799_-785916563 | \n", "2022-02-23 12:42:05.423489800+00:00 | \n", "efeb32222798ea6c2eb747514dd34e646cf751aa | \n", "win-6l54m49tpgn.woanware.co.uk | \n", "High | \n", "UnwantedSoftware | \n", "Notepad Execution Detection | \n", "notepad.exe | \n", "58bf532f93949cc62821273f778b87da85787afa | \n", "\n", " | \n", " | \n", " | 7173 | \n", "DeviceProcessEvents | \n", "
4 | \n", "ed637810396041195799_-785916563 | \n", "2022-02-21 12:50:38.523957+00:00 | \n", "efeb32222798ea6c2eb747514dd34e646cf751aa | \n", "win-6l54m49tpgn.woanware.co.uk | \n", "High | \n", "UnwantedSoftware | \n", "Notepad Execution Detection | \n", "notepad.exe | \n", "58bf532f93949cc62821273f778b87da85787afa | \n", "\n", " | \n", " | \n", " | 48272 | \n", "DeviceProcessEvents | \n", "
5 | \n", "da637831931152888965_11461089 | \n", "2022-03-18 09:36:44.691315900+00:00 | \n", "e09b459fcd3aa36f23084211c34bfca046848b65 | \n", "atevet06cl005.defenderatevet06.onmicrosoft.com | \n", "Medium | \n", "Execution | \n", "Suspicious PowerShell command line | \n", "powershell.exe | \n", "f43d9bb316e30ae1a3494ac5b0624f6bea1bf054 | \n", "\n", " | \n", " | [\"PowerShell (T1059.001)\"] | \n", "127261 | \n", "DeviceImageLoadEvents | \n", "
6 | \n", "da637823935780152927_1691659474 | \n", "2022-03-09 03:31:17.271601600+00:00 | \n", "e09b459fcd3aa36f23084211c34bfca046848b65 | \n", "atevet06cl005.defenderatevet06.onmicrosoft.com | \n", "Medium | \n", "Persistence | \n", "A WMI event filter was bound to a suspicious e... | \n", "\n", " | \n", " | \n", " | \n", " | [\"Windows Management Instrumentation Event Sub... | \n", "771 | \n", "DeviceEvents | \n", "
7 | \n", "da637819900757032477_-2012458047 | \n", "2022-03-04 11:20:44.426056800+00:00 | \n", "efeb32222798ea6c2eb747514dd34e646cf751aa | \n", "win-6l54m49tpgn.woanware.co.uk | \n", "Low | \n", "DefenseEvasion | \n", "Windows Sysinternals tool renamed | \n", "powershell.exe | \n", "5b1a36a8b206fe9daece7e2cd702cdd4e10bb895 | \n", "\n", " | \n", " | [\"Masquerading (T1036)\"] | \n", "43063 | \n", "DeviceProcessEvents | \n", "
8 | \n", "da637823935780204986_825228657 | \n", "2022-03-09 03:31:17.272119+00:00 | \n", "e09b459fcd3aa36f23084211c34bfca046848b65 | \n", "atevet06cl005.defenderatevet06.onmicrosoft.com | \n", "Medium | \n", "Persistence | \n", "A WMI event filter was bound to a suspicious e... | \n", "\n", " | \n", " | \n", " | \n", " | [\"Windows Management Instrumentation Event Sub... | \n", "774 | \n", "DeviceEvents | \n", "
9 | \n", "ed637828714417577685_889446803 | \n", "2022-03-14 16:03:11.918913300+00:00 | \n", "a085f9e3a00689554d351bdcdd4d503a1983eecd | \n", "desktop-6qf1uf2 | \n", "High | \n", "UnwantedSoftware | \n", "Notepad Execution Detection | \n", "notepad.exe | \n", "5b80bbb07b1a84384e61fb3f9366cad97904ebea | \n", "\n", " | \n", " | \n", " | 25654 | \n", "Not supported | \n", "