{ "metadata": { "name": "", "signature": "sha256:2dbb2f1330b2cb8ff4d81035e42c352fa78ae99b51ca379a2f1ea1aa0793a25f" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "# PCAP to Graph\n", "This notebook demonstrates how short and sweet a workbench python script can be. :)\n", "\n", "Here we're using the workbench server to look at a specific case captured by [ThreatGlass](http://www.threatglass.com/). The exploited website for this exercise is gold-xxx.net [ThreatGlass_Info](http://www.threatglass.com/malicious_urls/141deabbc8741175d9f51559cf4ef3dd?process_date=2014-05-29).\n", "\n", "**Tools in this Notebook:**\n", "\n", "- Workbench: Open Source Security Framework [Workbench GitHub](https://github.com/SuperCowPowers/workbench)\n", "- Bro Network Security Monitor (http://www.bro.org)\n", "\n", "**More Info:** \n", "\n", "- See [PCAP_to_Dataframe](http://nbviewer.ipython.org/github/SuperCowPowers/workbench/blob/master/workbench/notebooks/PCAP_to_Dataframe.ipynb) for a short notebook on turning this PCAP into a Pandas Dataframe.\n", "- See [Workbench Demo Notebook](http://nbviewer.ipython.org/github/SuperCowPowers/workbench/blob/master/workbench/notebooks/Workbench_Demo.ipynb) for a lot more info on using workbench.\n", "\n", "## Lets start up the workbench server...\n", "Run the workbench server (from somewhere, for the demo we're just going to start a local one)\n", "
\n",
      "$ workbench_server\n",
      "
" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# Lets start to interact with workbench, please note there is NO specific client to workbench,\n", "# Just use the ZeroRPC Python, Node.js, or CLI interfaces.\n", "import zerorpc\n", "c = zerorpc.Client(timeout=120)\n", "c.connect(\"tcp://127.0.0.1:4242\")" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 5, "text": [ "[None]" ] } ], "prompt_number": 5 }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "# Read in the Data\n", " The data is pulled from [ThreatGlass](http://www.threatglass.com/), the exploited website for this exercise is gold-xxx.net [ThreatGlass_Info](http://www.threatglass.com/malicious_urls/141deabbc8741175d9f51559cf4ef3dd?process_date=2014-05-29).\n", "" ] }, { "cell_type": "code", "collapsed": false, "input": [ "# Load in the PCAP file\n", "with open('../data/pcap/gold_xxx.pcap','rb') as f:\n", " pcap_md5 = c.store_sample(f.read(), 'gold_xxx', 'pcap')" ], "language": "python", "metadata": {}, "outputs": [], "prompt_number": 6 }, { "cell_type": "code", "collapsed": false, "input": [ "# Now give us a HTTP graph of all the activities within that PCAP.\n", "# Workbench also has DNS and CONN graphs, but for now we're just interested in HTTP.\n", "c.work_request('pcap_http_graph', pcap_md5)" ], "language": "python", "metadata": {}, "outputs": [ { "metadata": {}, "output_type": "pyout", "prompt_number": 7, "text": [ "{'pcap_http_graph': {'md5': 'c8e58ff22b9a8e48838373fbb1692bdd',\n", " 'output': 'go to http://localhost:7474/browser and execute this query \"match (s:origin), (t:file), p=allShortestPaths((s)--(t)) return p\"'}}" ] } ], "prompt_number": 7 }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "# Workbench + Neo4j = Awesome\n", " The HTTP graph has quite a bit of info, but you can see that we've conducted a shortest paths search from all nodes of type 'origin' (any node originating http communications) to any node of type 'file'. So for this use case we're interested in all of the various files that got downloaded from our network tap in the last few minutes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#Wrap Up\n", "Well for this short notebook we used 3 lines of python to go from PCAP file to Neo4j graph. We hope this exercise showed some neato functionality using [Workbench](https://github.com/SuperCowPowers/workbench), we encourage you to check out the GitHub repository and our other notebooks:\n", "\n", "- [PCAP_to_Dataframe](http://nbviewer.ipython.org/github/SuperCowPowers/workbench/blob/master/workbench/notebooks/PCAP_to_Dataframe.ipynb) for a short notebook on turning this PCAP into a Pandas Dataframe.\n", "- [Workbench Demo](http://nbviewer.ipython.org/url/raw.github.com/SuperCowPowers/workbench/master/workbench/notebooks/Workbench_Demo.ipynb) general introduction to Workbench.\n", "- [PCAP_DriveBy](http://nbviewer.ipython.org/url/raw.github.com/SuperCowPowers/workbench/master/workbench/notebooks/PCAP_DriveBy.ipynb) a detail look at a Web DriveBy from the [ThreatGlass](http://www.threatglass.com) repository.\n", "- [PE File Sim Graph](http://nbviewer.ipython.org/url/raw.github.com/SuperCowPowers/workbench/master/workbench/notebooks/PE_SimGraph.ipynb) using Neo4j to generate a similarity graph using PE File features.\n", "- [Generator Pipelines](http://nbviewer.ipython.org/url/raw.github.com/SuperCowPowers/workbench/master/workbench/notebooks/Generator_Pipelines.ipynb) using the client/server streaming generators to demonstrate 'chaining' generators." ] } ], "metadata": {} } ] }