{ "metadata": { "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.1-final" }, "orig_nbformat": 2, "kernelspec": { "name": "Python 3.8.1 64-bit", "display_name": "Python 3.8.1 64-bit", "metadata": { "interpreter": { "hash": "c7a650d791d0a1d035b66682f8967f04fed3045153a1ba3c3bfeefd2541b18a6" } } } }, "nbformat": 4, "nbformat_minor": 2, "cells": [ { "source": [ "# Correlation Heatmap" ], "cell_type": "markdown", "metadata": {} }, { "source": [ "" ], "cell_type": "markdown", "metadata": {} }, { "source": [ "In this notebook we will use the [netdata-pandas](https://github.com/netdata/netdata-pandas) Python package to pull some data from some demo Netdata servers and make some pretty looking correlation heatmaps, because we all love a good heatmap don't we. \n", "\n", "**Note**: you can click the \"Open in Colab\" button above to open this notebook in [Google Colab](https://colab.research.google.com/notebooks/intro.ipynb#recent=true) where you can just get going with it without having to set up python enviornments or any messy stuff like that." ], "cell_type": "markdown", "metadata": {} }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# if you need to, uncomment below to install netdata-pandas and seaborn packages\n", "#!pip install seaborn==0.11.0 netdata-pandas==0.0.27" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import pandas as pd\n", "import seaborn as sns\n", "import matplotlib.pyplot as plt\n", "from netdata_pandas.data import get_data" ] }, { "source": [ "Lets pull the data from three demo nodes for a subset of charts for the last 15 minutes." ], "cell_type": "markdown", "metadata": {} }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# inputs\n", "hosts = ['london.my-netdata.io', 'cdn77.my-netdata.io', 'octopuscs.my-netdata.io']\n", "charts = ['system.cpu', 'system.load', 'system.io', 'system.ram', 'system.net', 'system.ip', 'system.processes', 'system.intr', 'system.softnet_stat']\n", "before = 0\n", "after = -60*15" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "(2700, 30)\n" ] }, { "output_type": "execute_result", "data": { "text/plain": [ " system.cpu|guest system.cpu|guest_nice \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 0.0 0.0 \n", " 1603460666 0.0 0.0 \n", " 1603460667 0.0 0.0 \n", " 1603460668 0.0 0.0 \n", " 1603460669 0.0 0.0 \n", "\n", " system.cpu|iowait system.cpu|irq \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 32.82828 0.0 \n", " 1603460666 0.00000 0.0 \n", " 1603460667 0.00000 0.0 \n", " 1603460668 0.00000 0.0 \n", " 1603460669 0.00000 0.0 \n", "\n", " system.cpu|nice system.cpu|softirq \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 0.0 0.0 \n", " 1603460666 0.0 0.0 \n", " 1603460667 0.0 0.0 \n", " 1603460668 0.0 0.0 \n", " 1603460669 0.0 0.0 \n", "\n", " system.cpu|steal system.cpu|system \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 0.0 2.020202 \n", " 1603460666 0.0 1.507538 \n", " 1603460667 0.0 1.515151 \n", " 1603460668 0.0 1.015228 \n", " 1603460669 0.0 2.000000 \n", "\n", " system.cpu|user system.intr|interrupts ... \\\n", "host time_idx ... \n", "cdn77.my-netdata.io 1603460665 0.505050 339.1384 ... \n", " 1603460666 2.010050 366.6814 ... \n", " 1603460667 0.505050 334.6279 ... \n", " 1603460668 0.507614 307.5347 ... \n", " 1603460669 1.000000 324.9979 ... \n", "\n", " system.processes|running system.ram|buffers \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 2.0 244.9961 \n", " 1603460666 4.0 244.9961 \n", " 1603460667 1.0 244.9961 \n", " 1603460668 1.0 244.9961 \n", " 1603460669 4.0 244.9961 \n", "\n", " system.ram|cached system.ram|free \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 2650.066 392.9688 \n", " 1603460666 2650.066 389.1836 \n", " 1603460667 2650.070 389.2617 \n", " 1603460668 2650.070 389.2305 \n", " 1603460669 2650.066 388.8750 \n", "\n", " system.ram|used system.softnet_stat|dropped \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 677.4180 0.0 \n", " 1603460666 681.2031 0.0 \n", " 1603460667 681.1211 0.0 \n", " 1603460668 681.1523 0.0 \n", " 1603460669 681.5117 0.0 \n", "\n", " system.softnet_stat|flow_limit_count \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 0.0 \n", " 1603460666 0.0 \n", " 1603460667 0.0 \n", " 1603460668 0.0 \n", " 1603460669 0.0 \n", "\n", " system.softnet_stat|processed \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 185.18660 \n", " 1603460666 212.22230 \n", " 1603460667 192.34850 \n", " 1603460668 163.72629 \n", " 1603460669 186.40330 \n", "\n", " system.softnet_stat|received_rps \\\n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 0.0 \n", " 1603460666 0.0 \n", " 1603460667 0.0 \n", " 1603460668 0.0 \n", " 1603460669 0.0 \n", "\n", " system.softnet_stat|squeezed \n", "host time_idx \n", "cdn77.my-netdata.io 1603460665 0.0 \n", " 1603460666 0.0 \n", " 1603460667 0.0 \n", " 1603460668 0.0 \n", " 1603460669 0.0 \n", "\n", "[5 rows x 30 columns]" ], "text/html": "
\n | \n | system.cpu|guest | \nsystem.cpu|guest_nice | \nsystem.cpu|iowait | \nsystem.cpu|irq | \nsystem.cpu|nice | \nsystem.cpu|softirq | \nsystem.cpu|steal | \nsystem.cpu|system | \nsystem.cpu|user | \nsystem.intr|interrupts | \n... | \nsystem.processes|running | \nsystem.ram|buffers | \nsystem.ram|cached | \nsystem.ram|free | \nsystem.ram|used | \nsystem.softnet_stat|dropped | \nsystem.softnet_stat|flow_limit_count | \nsystem.softnet_stat|processed | \nsystem.softnet_stat|received_rps | \nsystem.softnet_stat|squeezed | \n
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
host | \ntime_idx | \n\n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n | \n |
cdn77.my-netdata.io | \n1603460665 | \n0.0 | \n0.0 | \n32.82828 | \n0.0 | \n0.0 | \n0.0 | \n0.0 | \n2.020202 | \n0.505050 | \n339.1384 | \n... | \n2.0 | \n244.9961 | \n2650.066 | \n392.9688 | \n677.4180 | \n0.0 | \n0.0 | \n185.18660 | \n0.0 | \n0.0 | \n
1603460666 | \n0.0 | \n0.0 | \n0.00000 | \n0.0 | \n0.0 | \n0.0 | \n0.0 | \n1.507538 | \n2.010050 | \n366.6814 | \n... | \n4.0 | \n244.9961 | \n2650.066 | \n389.1836 | \n681.2031 | \n0.0 | \n0.0 | \n212.22230 | \n0.0 | \n0.0 | \n|
1603460667 | \n0.0 | \n0.0 | \n0.00000 | \n0.0 | \n0.0 | \n0.0 | \n0.0 | \n1.515151 | \n0.505050 | \n334.6279 | \n... | \n1.0 | \n244.9961 | \n2650.070 | \n389.2617 | \n681.1211 | \n0.0 | \n0.0 | \n192.34850 | \n0.0 | \n0.0 | \n|
1603460668 | \n0.0 | \n0.0 | \n0.00000 | \n0.0 | \n0.0 | \n0.0 | \n0.0 | \n1.015228 | \n0.507614 | \n307.5347 | \n... | \n1.0 | \n244.9961 | \n2650.070 | \n389.2305 | \n681.1523 | \n0.0 | \n0.0 | \n163.72629 | \n0.0 | \n0.0 | \n|
1603460669 | \n0.0 | \n0.0 | \n0.00000 | \n0.0 | \n0.0 | \n0.0 | \n0.0 | \n2.000000 | \n1.000000 | \n324.9979 | \n... | \n4.0 | \n244.9961 | \n2650.066 | \n388.8750 | \n681.5117 | \n0.0 | \n0.0 | \n186.40330 | \n0.0 | \n0.0 | \n
5 rows × 30 columns
\n