zabbix_export: version: '5.4' date: '2021-11-21T21:54:02Z' groups: - uuid: 7df96b18c230490a9a0a9e2307226338 name: Templates templates: - uuid: 24af6fabd98d4495b0fe43775e9538d5 template: 'App Elasticsearch Cluster new' name: 'App Elasticsearch Cluster new' description: | ## Overview **ElasticSearch Zabbix monitoring** =================================== #### Script-free Zabbix ES monitoring This template monitores all ES cluster using Zabbix 4.x HTTP Agent resource. This allows check ES being OnPremise or PAAS (AWS Elasticsearch, for example) without additional scripts. ### Requisites: * ES available for Zabbix server or a Zabbix proxy. That's all. * ES Endpoints can be adjusted on template macro. ### **Discovers:** * ES Indexes discovery ES Node discovery ### **Monitored Items:** * Shards * Cluster Rate * Cluster Latency * Cluster Health * JVM Stats * Disk Status * Snapshot status * ES Port * Memory * Documents (searchable, deleted) ## Author Rickk Barbosa (https://github.com/rickkbarbosa) groups: - name: Templates items: - uuid: 1500772bb7cd4a9b947ba29d5620eeb2 name: 'Elasticsearch Memory (Average per Node)' type: CALCULATED key: 'elasticsearch.cluster.memory[total,pernode]' delay: 5m history: 1w value_type: FLOAT units: b params: 'last(//elasticsearch.memory[total,cluster]) / last(//elasticsearch.cluster[number_of_nodes])' description: 'Total memory (sum of all nodes)' tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Health' - uuid: fba45075546546599d32fb995933829a name: 'Elasticsearch - Number of active primary shards' type: DEPENDENT key: 'elasticsearch.cluster[active_primary_shards]' delay: '0' history: 1w preprocessing: - type: JSONPATH parameters: - $.active_primary_shards master_item: key: 'elasticsearch.cluster[all,health]' request_method: POST tags: - tag: Application value: 'ES Cluster' - tag: Application value: 'ES Shards' - uuid: 4f689684db11489da1532f15bb5dea5b name: 'Elasticsearch - Number of active shards' type: DEPENDENT key: 'elasticsearch.cluster[active_shards]' delay: '0' history: 1w preprocessing: - type: JSONPATH parameters: - $.active_shards master_item: key: 'elasticsearch.cluster[all,health]' request_method: POST tags: - tag: Application value: 'ES Cluster' - tag: Application value: 'ES Shards' - uuid: f34f6c2add2344779d72e6220952c997 name: 'Elasticsearch Cluster Health' type: HTTP_AGENT key: 'elasticsearch.cluster[all,health]' history: 1d trends: '0' value_type: TEXT timeout: 10s url: '{$ELASTICSEARCH_PROTOCOL}://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_cluster/health' tags: - tag: Application value: 'ES Cluster' - uuid: 2feb30c48b1c4e3faba01c08c6bfa69c name: 'Elasticsearch Cluster Global Status' type: HTTP_AGENT key: 'elasticsearch.cluster[all,stats]' history: 1d trends: '0' value_type: TEXT timeout: 5s url: '{$ELASTICSEARCH_PROTOCOL}://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_cluster/stats' tags: - tag: Application value: 'ES Cluster' - uuid: 78322b5f43cd4d958e7957cad564a25a name: 'Elasticsearch - Number of data nodes' type: DEPENDENT key: 'elasticsearch.cluster[cluster,number_of_data_nodes]' delay: '0' history: 1w preprocessing: - type: JSONPATH parameters: - $.number_of_data_nodes master_item: key: 'elasticsearch.cluster[all,health]' request_method: POST tags: - tag: Application value: 'ES Cluster' - uuid: 20ae99e0861b4d6d8eb83eb5a45d3ada name: 'Master instance connection status' type: DEPENDENT key: 'elasticsearch.cluster[discovered_master]' delay: '0' history: 1w description: | Master instance connection status. Indicates whether data nodes can reach the master node. Failures are usually the result of a network connectivity problem. valuemap: name: Boolean preprocessing: - type: JSONPATH parameters: - $.discovered_master - type: REGEX parameters: - 'true' - '1' error_handler: CUSTOM_ERROR error_handler_params: '0' master_item: key: 'elasticsearch.cluster[all,health]' tags: - tag: Application value: 'ES Health' - uuid: 6ddabb586e924b2db0a07fa24e6f63b3 name: 'Elasticsearch - Number of initializing shards' type: DEPENDENT key: 'elasticsearch.cluster[initializing_shards]' delay: '0' history: 1w preprocessing: - type: JSONPATH parameters: - $.initializing_shards master_item: key: 'elasticsearch.cluster[all,health]' request_method: POST tags: - tag: Application value: 'ES Cluster' - tag: Application value: 'ES Shards' - uuid: 1e17a3c226ef469bbd432266eeb72654 name: 'Elasticsearch - Cluster Name' type: DEPENDENT key: 'elasticsearch.cluster[name]' delay: '0' history: 1w trends: '0' value_type: TEXT preprocessing: - type: JSONPATH parameters: - $.cluster_name master_item: key: 'elasticsearch.cluster[all,health]' tags: - tag: Application value: 'ES Cluster' - uuid: 9586d5c617f14e59ae292a3143f1af1e name: 'Number of nodes' type: DEPENDENT key: 'elasticsearch.cluster[number_of_nodes]' delay: '0' history: 1w preprocessing: - type: JSONPATH parameters: - $.number_of_nodes master_item: key: 'elasticsearch.cluster[all,health]' request_method: POST tags: - tag: Application value: 'ES Cluster' - uuid: b20a72c5e8ee4615ab6f72fdbe787647 name: 'Elasticsearch - Number of relocating shards' type: DEPENDENT key: 'elasticsearch.cluster[relocating_shards]' delay: '0' history: 1w preprocessing: - type: JSONPATH parameters: - $.relocating_shards master_item: key: 'elasticsearch.cluster[all,health]' request_method: POST tags: - tag: Application value: 'ES Cluster' - tag: Application value: 'ES Shards' - uuid: 5d80a6fe865249e2894f7bdaed7ff57a name: 'Elasticsearch - Cluster Size' type: DEPENDENT key: 'elasticsearch.cluster[size]' delay: '0' history: 1w units: b description: 'Total cluster size in bytes' preprocessing: - type: JSONPATH parameters: - $.indices.store.size_in_bytes master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES Cluster' - uuid: 699f66e8b9a942598fb121029efe1263 name: 'Elasticsearch - Cluster Status' type: DEPENDENT key: 'elasticsearch.cluster[status]' delay: '0' history: 1w trends: '0' value_type: TEXT preprocessing: - type: JSONPATH parameters: - $.status master_item: key: 'elasticsearch.cluster[all,health]' request_method: POST tags: - tag: Application value: 'ES Cluster' triggers: - uuid: e75cf3cf39ef4b91ac368ababe1d43ec expression: 'find(/App Elasticsearch Cluster new/elasticsearch.cluster[status],3s,"iregexp","green")=0' recovery_mode: RECOVERY_EXPRESSION recovery_expression: 'find(/App Elasticsearch Cluster new/elasticsearch.cluster[status],3s,"iregexp","green")=1' name: '[ {HOST.NAME} ] - Elasticsearch Cluster in {ITEM.LASTVALUE} state' priority: WARNING description: 'The cluster health status is: green, yellow or red. On the shard level, a red status indicates that the specific shard is not allocated in the cluster, yellow means that the primary shard is allocated but replicas are not, and green means that all shards are allocated. The index level status is controlled by the worst shard status. The cluster status is controlled by the worst index status.' dependencies: - name: '[ {HOST.NAME} ] - Elasticsearch Cluster in {ITEM.LASTVALUE} state' expression: 'find(/App Elasticsearch Cluster new/elasticsearch.cluster[status],5s,"iregexp","green")=0' recovery_expression: 'find(/App Elasticsearch Cluster new/elasticsearch.cluster[status],3s,"iregexp","green")=1' - uuid: 7a03899152044dcda018cc5be63d7245 expression: 'find(/App Elasticsearch Cluster new/elasticsearch.cluster[status],5s,"iregexp","green")=0' recovery_mode: RECOVERY_EXPRESSION recovery_expression: 'find(/App Elasticsearch Cluster new/elasticsearch.cluster[status],3s,"iregexp","green")=1' name: '[ {HOST.NAME} ] - Elasticsearch Cluster in {ITEM.LASTVALUE} state' priority: AVERAGE description: 'The cluster health status is: green, yellow or red. On the shard level, a red status indicates that the specific shard is not allocated in the cluster, yellow means that the primary shard is allocated but replicas are not, and green means that all shards are allocated. The index level status is controlled by the worst shard status. The cluster status is controlled by the worst index status.' - uuid: a94dc1bdb07949909e381a0074f32bbd expression: 'nodata(/App Elasticsearch Cluster new/elasticsearch.cluster[status],5m)=1' recovery_mode: RECOVERY_EXPRESSION recovery_expression: 'nodata(/App Elasticsearch Cluster new/elasticsearch.cluster[status],3m)=0' name: '[ {HOST.NAME} ] - Elasticsearch Monitoring is not collecting data' priority: AVERAGE dependencies: - name: '[ {HOST.NAME} ] - Elasticsearch Port is unavailable' expression: 'sum(/App Elasticsearch Cluster new/net.tcp.service[tcp,{$ELASTICSEARCH_HOST},{$ELASTICSEARCH_PORT}],#3)=0' recovery_expression: 'avg(/App Elasticsearch Cluster new/net.tcp.service[tcp,{$ELASTICSEARCH_HOST},{$ELASTICSEARCH_PORT}],#3)=1' - uuid: f2ea7fb11ed44f158bb431715c442f0f name: 'Elasticsearch - Number of unassigned shards' type: DEPENDENT key: 'elasticsearch.cluster[unassigned_shards]' delay: '0' history: 1w preprocessing: - type: JSONPATH parameters: - $.unassigned_shards master_item: key: 'elasticsearch.cluster[all,health]' request_method: POST tags: - tag: Application value: 'ES Cluster' - tag: Application value: 'ES Shards' - uuid: 1ee65414cbae4156a4cc7a380c3e183a name: 'Elasticsearch Cluster UUID' type: DEPENDENT key: 'elasticsearch.cluster[uuid]' delay: '0' history: 1w trends: '0' value_type: TEXT preprocessing: - type: JSONPATH parameters: - $.cluster_uuid master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES Cluster' - tag: Application value: 'ES General status' - uuid: 5861f34425704489add5ecef5503d7d1 name: 'Elasticsearch CPU Usage' type: DEPENDENT key: elasticsearch.cpu delay: '0' history: 1w value_type: FLOAT units: '%' description: 'CPU Usage in percent on Cluster. It checks all node.' preprocessing: - type: JSONPATH parameters: - $.nodes.process.cpu.percent master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES Cluster' - tag: Application value: 'ES Health' - uuid: c85121a57ffd4d0cae1d9240b1df552e name: 'Deleted documents' type: DEPENDENT key: elasticsearch.deleted delay: '0' history: 1d description: 'Total Number of Records marked for deletion' preprocessing: - type: JSONPATH parameters: - $.indices.docs.deleted master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES Data' - uuid: fbcfbfe18c83413da0d9eaf22fb7ea5e name: 'Elasticsearch Disk Volume' type: HTTP_AGENT key: 'elasticsearch.disk[all]' delay: 10m history: 1d trends: '0' value_type: TEXT timeout: 10s url: '{$ELASTICSEARCH_PROTOCOL}://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_nodes/stats/fs' tags: - tag: Application value: 'ES General status' - uuid: 3ce4756ada1b4331a5a0404424e179ec name: 'Elasticsearch Disk Free (%)' type: CALCULATED key: 'elasticsearch.disk[free,percent]' delay: 5m history: 1w value_type: FLOAT units: '%' params: '( last(//elasticsearch.disk[free]) / last(//elasticsearch.disk[total]) ) * 100' description: 'Free disk volume (in percent)' tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Health' - uuid: 062b7f03b91f424ebb624d2cef5aa4b8 name: 'Elasticsearch Disk Volume Free' type: DEPENDENT key: 'elasticsearch.disk[free]' delay: '0' history: 1d units: b preprocessing: - type: JSONPATH parameters: - $.nodes.fs.total.free_in_bytes master_item: key: 'elasticsearch.disk[all]' tags: - tag: Application value: 'ES Health' - uuid: b021aceaaca44bf7826c8f1416e47a77 name: 'Elasticsearch Disk Volume Total' type: DEPENDENT key: 'elasticsearch.disk[total]' delay: '0' history: 1d units: b preprocessing: - type: JSONPATH parameters: - $.nodes.fs.total.total_in_bytes master_item: key: 'elasticsearch.disk[all]' tags: - tag: Application value: 'ES Health' - uuid: 6969b09a076540a18683069e4c4c12ef name: 'Elasticsearch Indices Global Status' type: HTTP_AGENT key: 'elasticsearch.indices[all,stats]' history: 1d trends: '0' value_type: TEXT description: | Indices level stats provide statistics on different operations happening on an index. The API provides statistics on the index level scope (though most stats can also be retrieved using node level scope). Base for key performance indicator https://www.elastic.co/guide/en/elasticsearch/reference/6.4/indices-stats.html timeout: 10s url: '{$ELASTICSEARCH_PROTOCOL}://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_stats' tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Key performance indicators' - uuid: 3f5fc095d35940e2824584fbc0b71853 name: 'Elasticsearch JVM Heap (Max)' type: DEPENDENT key: 'elasticsearch.jvm[heap,max]' delay: '0' history: 1w units: b description: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html' preprocessing: - type: JSONPATH parameters: - $.nodes.jvm.mem.heap_max_in_bytes master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES JVM Stats' - uuid: 3a3b30026c234cc683602a4d23154e0e name: 'Elasticsearch JVM Heap (Used, Percent)' type: CALCULATED key: 'elasticsearch.jvm[heap,usedp]' delay: 5m history: 1w value_type: FLOAT units: '%' params: '( last(//elasticsearch.jvm[heap,used]) / last(//elasticsearch.jvm[heap,max]) ) *100' description: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html' tags: - tag: Application value: 'ES JVM Stats' triggers: - uuid: 4e2694a607b6414d968160e216fb9cfc expression: 'avg(/App Elasticsearch Cluster new/elasticsearch.jvm[heap,usedp],#3)>{$ELASTICSEARCH_HEAPMEM_P2}' name: '[ {HOST.NAME} ] - Elasticsearch Heap Memory Used is over {$ELASTICSEARCH_HEAPMEM_P2}' priority: AVERAGE - uuid: 973b950e7a284ddd9ba76657e93eeefd name: 'Elasticsearch JVM Heap (Used, bytes)' type: DEPENDENT key: 'elasticsearch.jvm[heap,used]' delay: '0' history: 1w units: b description: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html' preprocessing: - type: JSONPATH parameters: - $.nodes.jvm.mem.heap_used_in_bytes master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES JVM Stats' - uuid: ccb5d16876af4e6e8dc5b2c88c0fd4ec name: 'Elasticsearch JVM Version' type: DEPENDENT key: 'elasticsearch.jvm[version]' delay: '0' history: 1w trends: '0' value_type: TEXT preprocessing: - type: JSONPATH parameters: - $.nodes.jvm.versions master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES JVM Stats' - uuid: b09613ac151947e1b59f09dacd873d0a name: 'Elasticsearch Memory Free (%)' type: CALCULATED key: 'elasticsearch.memory[free,cluster,percentage]' delay: 5m history: 1w value_type: FLOAT units: '%' params: '( last(//elasticsearch.memory[free,cluster]) / last(//elasticsearch.memory[total,cluster]) ) * 100' description: 'Free memory in cluster (in percent)' tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Health' - uuid: b1d3f68689fd42da8fa8678a43564446 name: 'Elasticsearch Memory Free' type: DEPENDENT key: 'elasticsearch.memory[free,cluster]' delay: '0' history: 1w value_type: FLOAT units: b description: 'Free memory on cluster (sum of all nodes)' preprocessing: - type: JSONPATH parameters: - $.nodes.os.mem.free_in_bytes master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Health' - uuid: a3fc5ca137654e3dbf53909614695a47 name: 'Elasticsearch Memory (Cluster)' type: DEPENDENT key: 'elasticsearch.memory[total,cluster]' delay: '0' history: 1w value_type: FLOAT units: b description: 'Total memory (sum of all nodes)' preprocessing: - type: JSONPATH parameters: - $.nodes.os.mem.total_in_bytes master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Health' - uuid: 959d64e0313f4a14983eea4038fd8598 name: 'Elasticsearch - Indexing rate' type: DEPENDENT key: 'elasticsearch.performance[index]' delay: '0' history: 1w value_type: FLOAT units: ops/min description: 'Number of index operations per minute.' preprocessing: - type: JSONPATH parameters: - $._all.primaries.indexing.index_total - type: SIMPLE_CHANGE parameters: - '' master_item: key: 'elasticsearch.indices[all,stats]' tags: - tag: Application value: 'ES Health' - tag: Application value: 'ES Key performance indicators' - uuid: 46a7761b68de4b8d93b0791ab3386910 name: 'Elasticsearch - Indexing latency (ms)' type: DEPENDENT key: 'elasticsearch.performance[latency,index]' delay: '0' history: 1w value_type: FLOAT units: ms description: 'Average time that it takes a shard to complete and indexing operation' preprocessing: - type: JSONPATH parameters: - $._all.primaries.indexing.index_time_in_millis - type: CHANGE_PER_SECOND parameters: - '' master_item: key: 'elasticsearch.indices[all,stats]' tags: - tag: Application value: 'ES Health' - tag: Application value: 'ES Key performance indicators' - uuid: 8de19369259347fc9e063e4b7f53700c name: 'Elasticsearch - Search latency (ms)' type: DEPENDENT key: 'elasticsearch.performance[latency,search]' delay: '0' history: 1w value_type: FLOAT units: ms description: 'Average time that takes a shard to complete a search operation' preprocessing: - type: JSONPATH parameters: - $._all.primaries.search.query_time_in_millis - type: CHANGE_PER_SECOND parameters: - '' master_item: key: 'elasticsearch.indices[all,stats]' tags: - tag: Application value: 'ES Health' - tag: Application value: 'ES Key performance indicators' - uuid: 665c2b78ffe443f6a75f5b2e7ab69f36 name: 'Elasticsearch - Search rate' type: DEPENDENT key: 'elasticsearch.performance[search]' delay: '0' history: 1w value_type: FLOAT units: ops/min description: 'Search operations per minute.' preprocessing: - type: JSONPATH parameters: - $._all.primaries.search.query_total - type: SIMPLE_CHANGE parameters: - '' master_item: key: 'elasticsearch.indices[all,stats]' tags: - tag: Application value: 'ES Health' - tag: Application value: 'ES Key performance indicators' - uuid: ad6c123388c94df5bb0ac95b6715fe84 name: 'Searchable documents' type: DEPENDENT key: elasticsearch.records delay: '0' history: 1d description: 'Total Number of Records' preprocessing: - type: JSONPATH parameters: - $.indices.docs.count master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES Data' - uuid: 652e693c86024e9bb8fea6111ac2767f name: 'Elasticsearch Well-done Snapshots in last {$ELASTICSEARCH_SNAPSHOTP_DAYS} days' type: DEPENDENT key: 'elasticsearch.snapshots[ok]' delay: '0' history: 1w description: 'Total snapshots in the last {$ELASTICSEARCH_SNAPSHOTP_DAYS} days that has been succeed' valuemap: name: Boolean preprocessing: - type: REGEX parameters: - ',([0-9]+)' - \0 - type: TRIM parameters: - ',' master_item: key: 'elasticsearch.snapshots[stats]' tags: - tag: Application value: 'ES Health' - uuid: 0971da2b5241435b8cf68b1db75c61ad name: 'ES Snapshot Status' type: HTTP_AGENT key: 'elasticsearch.snapshots[stats]' delay: 1h history: 1d trends: '0' value_type: TEXT description: | A snapshot is a backup taken from a running Elasticsearch cluster. This presents how many snapshots exists in the last 3 days and how many has been succeed. https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html preprocessing: - type: JAVASCRIPT parameters: - | var lld = []; var lines = value.split("\n"); var lines_num = lines.length; //Date var days = {$ELASTICSEARCH_SNAPSHOTP_DAYS} days = 86400 * days //daysAgo = 86400 //3 days const now = new Date() const date = (Math.floor(Date.now() / 1000) - daysAgo) //Fetch last 3 days var output = " "; for (i = 0; i < lines_num; i++) { var line = lines[i].split(" ")[2]; if (line > date) { output = output + "\n" + lines[i]; } }//Regex var re = /SUCCESS/g, success = 0; while (re.exec(output) !== null) { ++success; } var total = output.split("\n"); var total = total.length - 1; result = total + "," + success; return result; timeout: 10s url: '{$ELASTICSEARCH_PROTOCOL}://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_cat/snapshots/{$ELASTICSEARCH_SNAPSHOT}' tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Health' - uuid: baf0b75d06db47a4ae05c583ef49ad2e name: 'Elasticsearch Snapshots in last {$ELASTICSEARCH_SNAPSHOTP_DAYS} days' type: DEPENDENT key: 'elasticsearch.snapshots[total]' delay: '0' history: 1w description: 'Total snapshots in the last 3 days' valuemap: name: Boolean preprocessing: - type: REGEX parameters: - '^([0-9]+),' - \0 - type: RTRIM parameters: - ',' master_item: key: 'elasticsearch.snapshots[stats]' tags: - tag: Application value: 'ES Health' - uuid: eda2977302cf4d3fbd86de5179836a7a name: 'Elasticsearch version' type: DEPENDENT key: elasticsearch.version delay: '0' history: 1w trends: '0' value_type: TEXT preprocessing: - type: JSONPATH parameters: - '$.nodes.versions[0]' master_item: key: 'elasticsearch.cluster[all,stats]' tags: - tag: Application value: 'ES General status' - uuid: 6ca749e6a0424461b513bb72a268617c name: 'Elasticsearch port listen' type: SIMPLE key: 'net.tcp.service[tcp,{$ELASTICSEARCH_HOST},{$ELASTICSEARCH_PORT}]' history: 1w valuemap: name: 'Service state' request_method: POST tags: - tag: Application value: 'ES Health' triggers: - uuid: 0956269f19864c68aff905a3c63d09b2 expression: 'sum(/App Elasticsearch Cluster new/net.tcp.service[tcp,{$ELASTICSEARCH_HOST},{$ELASTICSEARCH_PORT}],#3)=0' recovery_mode: RECOVERY_EXPRESSION recovery_expression: 'avg(/App Elasticsearch Cluster new/net.tcp.service[tcp,{$ELASTICSEARCH_HOST},{$ELASTICSEARCH_PORT}],#3)=1' name: '[ {HOST.NAME} ] - Elasticsearch Port is unavailable' priority: AVERAGE discovery_rules: - uuid: 8b29a77e62cc4c8692ffd2e462a70878 name: 'ES Indexes discovery' type: HTTP_AGENT key: elasticsearch.discovery.indexes filter: conditions: - macro: '{#ELASTICSEARCH_INDEX}' value: '^(?!\s*$).+' formulaid: A lifetime: 7d description: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-indices.html' item_prototypes: - uuid: 2263e6eb0f954c52ae8748fd8a92d42c name: 'Elasticsearch index full info [ {#ELASTICSEARCH_INDEX} ]' type: HTTP_AGENT key: 'elasticsearch.index[all,{#ELASTICSEARCH_INDEX}]' history: 1d trends: '0' value_type: TEXT preprocessing: - type: JAVASCRIPT parameters: - | var lld = []; var data = value.split(" "); var row = {}; row["ELASTICSEARCH_INDEX_HEALTH"] = data[0]; row["ELASTICSEARCH_INDEX_STATUS"] = data[1]; row["ELASTICSEARCH_INDEX_NAME"] = data[2]; row["ELASTICSEARCH_INDEX_UUID"] = data[3]; row["ELASTICSEARCH_INDEX_DOCSCOUNT"] = data[6]; row["ELASTICSEARCH_INDEX_DOCSDELETED"] = data[7]; row["ELASTICSEARCH_INDEX_SIZE"] = data[8]; row["ELASTICSEARCH_INDEX_PSIZE"] = data[9]; lld.push(row); return JSON.stringify(lld); - type: REGEX parameters: - '.*' - '{"data":\0' - type: REGEX parameters: - '.*' - '\0}' timeout: 10s url: 'http://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_cat/indices/{#ELASTICSEARCH_INDEX}' query_fields: - name: bytes value: b tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Indexes' - uuid: 636c1453a31f492b877f93dd3dbb2473 name: 'Elasticsearch index documents [ {#ELASTICSEARCH_INDEX} ]' type: DEPENDENT key: 'elasticsearch.index[documents,{#ELASTICSEARCH_INDEX}]' delay: '0' history: 7d preprocessing: - type: JSONPATH parameters: - '$.data[0].ELASTICSEARCH_INDEX_DOCSCOUNT' master_item: key: 'elasticsearch.index[all,{#ELASTICSEARCH_INDEX}]' tags: - tag: Application value: 'ES Indexes' - uuid: c695e5fcca4a4dfd9ed45f042b0ab499 name: 'Elasticsearch index documents deleted [ {#ELASTICSEARCH_INDEX} ]' type: DEPENDENT key: 'elasticsearch.index[documentsdeleted,{#ELASTICSEARCH_INDEX}]' delay: '0' history: 7d preprocessing: - type: JSONPATH parameters: - '$.data[0].ELASTICSEARCH_INDEX_DOCSDELETED' master_item: key: 'elasticsearch.index[all,{#ELASTICSEARCH_INDEX}]' tags: - tag: Application value: 'ES Indexes' - uuid: 630f704fb1af4f6cbce798395c4879a1 name: 'Elasticsearch index health [ {#ELASTICSEARCH_INDEX} ]' type: DEPENDENT key: 'elasticsearch.index[health,{#ELASTICSEARCH_INDEX}]' delay: '0' history: 7d trends: '0' value_type: TEXT preprocessing: - type: JSONPATH parameters: - '$.data[0].ELASTICSEARCH_INDEX_HEALTH' master_item: key: 'elasticsearch.index[all,{#ELASTICSEARCH_INDEX}]' tags: - tag: Application value: 'ES Indexes' - uuid: edc0e51cffbc4c39841fa02513f8178a name: 'Elasticsearch index latency [ {#ELASTICSEARCH_INDEX} ] (ms)' type: DEPENDENT key: 'elasticsearch.index[latency,{#ELASTICSEARCH_INDEX}]' delay: '0' history: 7d value_type: FLOAT units: ms description: | Average time that takes a shard to complete a search operation. Specific for a index. preprocessing: - type: JSONPATH parameters: - '$.indices.{#ELASTICSEARCH_INDEX}.indexing.index_time_in_millis' - type: CHANGE_PER_SECOND parameters: - '' master_item: key: 'elasticsearch.indices[all,stats]' tags: - tag: Application value: 'ES Indexes' - tag: Application value: 'ES Key performance indicators' - uuid: 997f004795ec4c54a73cda42c7b3c06d name: 'Elasticsearch queries [ {#ELASTICSEARCH_INDEX} ]' type: DEPENDENT key: 'elasticsearch.index[queries,{#ELASTICSEARCH_INDEX}]' delay: '0' history: 7d value_type: FLOAT description: 'Number of queries on this index' preprocessing: - type: JSONPATH parameters: - '$.indices.{#ELASTICSEARCH_INDEX}.search.query_total' - type: SIMPLE_CHANGE parameters: - '' master_item: key: 'elasticsearch.indices[all,stats]' tags: - tag: Application value: 'ES Indexes' - tag: Application value: 'ES Key performance indicators' - uuid: b5fc6a3318a4452fbc6b51d54552b047 name: 'Elasticsearch index query latency [ {#ELASTICSEARCH_INDEX} ]' type: DEPENDENT key: 'elasticsearch.index[querylatency,{#ELASTICSEARCH_INDEX}]' delay: '0' history: 7d value_type: FLOAT units: ms description: 'Search time in this index' preprocessing: - type: JSONPATH parameters: - '$.indices.{#ELASTICSEARCH_INDEX}.search.query_time_in_millis' - type: CHANGE_PER_SECOND parameters: - '' master_item: key: 'elasticsearch.indices[all,stats]' tags: - tag: Application value: 'ES Indexes' - tag: Application value: 'ES Key performance indicators' - uuid: 7678a26b0fd7455f9f77a7848c3ab766 name: 'Elasticsearch index size [ {#ELASTICSEARCH_INDEX} ]' type: DEPENDENT key: 'elasticsearch.index[size,{#ELASTICSEARCH_INDEX}]' delay: '0' history: 7d units: b preprocessing: - type: JSONPATH parameters: - '$.data[0].ELASTICSEARCH_INDEX_SIZE' master_item: key: 'elasticsearch.index[all,{#ELASTICSEARCH_INDEX}]' tags: - tag: Application value: 'ES Indexes' timeout: 10s url: '{$ELASTICSEARCH_PROTOCOL}://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_cat/indices' query_fields: - name: h value: index preprocessing: - type: JAVASCRIPT parameters: - | var lld = []; var lines = value.split("\n"); var lines_num = lines.length; for (i = 0; i < lines_num; i++) { var row = {}; row["{#ELASTICSEARCH_INDEX}"] = lines[i] lld.push(row); } return JSON.stringify(lld); - type: REGEX parameters: - '.*' - '{"data":\0' - type: REGEX parameters: - '.*' - '\0}' - uuid: 3c30043bfc4443199ba3969cb31e83a2 name: 'ES Node discovery' type: HTTP_AGENT key: elasticsearch.discovery.nodes filter: conditions: - macro: '{#ELASTICSEARCH_NODE}' value: '^(?!\s*$).+' formulaid: A lifetime: 7d description: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-nodes.html' item_prototypes: - uuid: 8d4ca69012b043719476e826c5050f81 name: 'Elasticsearch full allocation info [ {#ELASTICSEARCH_NODE} ]' type: HTTP_AGENT key: 'elasticsearch.node.disk[all,{#ELASTICSEARCH_NODE}]' history: 1d trends: '0' value_type: TEXT description: | Provides a snapshot of the number of shards allocated to each data node and their disk space. https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-allocation.html timeout: 1m url: 'http://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_cat/allocation/{#ELASTICSEARCH_NODE}' query_fields: - name: bytes value: b tags: - tag: Application value: 'ES Nodes' - uuid: ad77771a7c1a43fa8e9bc5ab76cd4c1e name: 'Elasticsearch node [ {#ELASTICSEARCH_NODE} ] is master?' type: HTTP_AGENT key: 'elasticsearch.node.master[{#ELASTICSEARCH_NODE}]' history: 1d trends: '0' value_type: TEXT description: | Get information about master node. https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-master.html preprocessing: - type: REGEX parameters: - '{#ELASTICSEARCH_NODE}' - '1' error_handler: CUSTOM_VALUE error_handler_params: '0' timeout: 1m url: 'http://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_cat/master' query_fields: - name: h value: node tags: - tag: Application value: 'ES Nodes' - uuid: c9e3464a0dcd4d2eac9662f9938f6c00 name: 'Elasticsearch full stats for node [ {#ELASTICSEARCH_NODE} ]' type: HTTP_AGENT key: 'elasticsearch.node.query_cache[all,{#ELASTICSEARCH_NODE}]' history: 1d trends: '0' value_type: TEXT description: | Full stats for specific node as seen on https://www.elastic.co/guide/en/elasticsearch/reference/6.2/cluster-nodes-stats.html timeout: 1m url: 'http://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_nodes/{#ELASTICSEARCH_NODE}/stats' tags: - tag: Application value: 'ES General status' - tag: Application value: 'ES Nodes' - uuid: 0d3d926fabd54bd384a65fede4d74507 name: 'Elasticsearch CPU Load (1min) [ {#ELASTICSEARCH_NODE} ]' type: DEPENDENT key: 'elasticsearch.node[cpu1m,{#ELASTICSEARCH_NODE}]' delay: '0' history: 7d trends: '0' value_type: TEXT preprocessing: - type: JSONPATH parameters: - $.nodes master_item: key: 'elasticsearch.node.query_cache[all,{#ELASTICSEARCH_NODE}]' tags: - tag: Application value: 'ES Health' - tag: Application value: 'ES Nodes' - uuid: c5a4c722b8914cf196f12e2126e87047 name: 'Elasticsearch Storage Total [ {#ELASTICSEARCH_NODE} ]' type: DEPENDENT key: 'elasticsearch.node[disk,{#ELASTICSEARCH_NODE},total]' delay: '0' history: 7d units: b description: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-allocation.html' preprocessing: - type: REGEX parameters: - '(?:(\d+)( )(\d+)( )((\d+|x)\.))' - \0 - type: REGEX parameters: - '^([0-9]+)' - \0 master_item: key: 'elasticsearch.node.disk[all,{#ELASTICSEARCH_NODE}]' tags: - tag: Application value: 'ES Nodes' - uuid: 384e63f4c3924232b5e34e256b05fb5f name: 'Elasticsearch Storage Used (in %) [ {#ELASTICSEARCH_NODE} ]' type: DEPENDENT key: 'elasticsearch.node[disk,{#ELASTICSEARCH_NODE},usedp]' delay: '0' history: 7d value_type: FLOAT units: '%' description: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-allocation.html' preprocessing: - type: REGEX parameters: - '(?:(\d+)( )(\d+)( )((\d+|x)\.))' - \0 - type: REGEX parameters: - '(( )[0-9]+)' - \0 master_item: key: 'elasticsearch.node.disk[all,{#ELASTICSEARCH_NODE}]' tags: - tag: Application value: 'ES Nodes' - uuid: 054d9854722f4f6489502bd851aac575 name: 'Elasticsearch Storage Used [ {#ELASTICSEARCH_NODE} ]' type: DEPENDENT key: 'elasticsearch.node[disk,{#ELASTICSEARCH_NODE},used]' delay: '0' history: 7d value_type: FLOAT units: b description: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-allocation.html' preprocessing: - type: REGEX parameters: - '(?:( )(\d+)( )(\d+))' - \0 - type: REGEX parameters: - '([0-9]+)$' - \0 master_item: key: 'elasticsearch.node.disk[all,{#ELASTICSEARCH_NODE}]' tags: - tag: Application value: 'ES Nodes' timeout: 10s url: '{$ELASTICSEARCH_PROTOCOL}://{$ELASTICSEARCH_HOST}:{$ELASTICSEARCH_PORT}/_cat/nodes' query_fields: - name: h value: name preprocessing: - type: JAVASCRIPT parameters: - | var lld = []; var lines = value.split("\n"); var lines_num = lines.length; for (i = 0; i < lines_num; i++) { var row = {}; row["{#ELASTICSEARCH_NODE}"] = lines[i] lld.push(row); } return JSON.stringify(lld); - type: REGEX parameters: - '.*' - '{"data":\0' - type: REGEX parameters: - '.*' - '\0}' macros: - macro: '{$ELASTICSEARCH_HEAPMEM_P2}' value: '75' - macro: '{$ELASTICSEARCH_HOST}' value: localhost - macro: '{$ELASTICSEARCH_PORT}' value: '9200' - macro: '{$ELASTICSEARCH_PROTOCOL}' value: http - macro: '{$ELASTICSEARCH_SNAPSHOT}' value: cs-automated-enc - macro: '{$ELASTICSEARCH_SNAPSHOTP_DAYS}' value: '3' valuemaps: - uuid: 4ed04ef08c4d4d3c991b991bf0cfbfe3 name: Boolean mappings: - value: 'False' newvalue: '0' - value: 'false' newvalue: '0' - value: 'True' newvalue: '1' - value: 'true' newvalue: '1' - uuid: 666b99a715f74bd9bfca93c684b9fa4b name: 'Service state' mappings: - value: '0' newvalue: Down - value: '1' newvalue: Up