Another groovy operations dashboard by Richard Morgan productions 62 (index=core_splunk_internal OR index=_internal) $_internal$ sourcetype=splunkd $selected_targets$ replication queue for TERM(peer=*) (BucketReplicator full) OR (has room now) | join peer [| loadjob $guid_to_host_mapping$] | rex field=_raw "bid=(?<idx>[^~]+)" | eval state=if(searchmatch("BucketReplicator full"), "blocked", "unblocked"), end=if(searchmatch("has room now"), true(), NULL) | table _time host bid peer _raw indexer idx state $time.earliest$ $time.latest$ 1 $job.sid$ | dbinspect index=_internal | stats count by guId splunk_server | rename splunk_server as indexer | rename guId as peer -15m now $job.sid$ $job.sid$ | loadjob $bucketreplicator_sid$ | search idx=$selected_idx$ | transaction host bid startswith=eval(state="blocked") endswith=eval(state="unblocked") mvraw=true $earliest$ $latest$
label search $_internal$ sourcetype=splunkd CMMaster status=success site* earliest=-4hr latest=now | rex field=_raw max_match=64 "(?<site_pair>site\d+,\"?[^\",]+)" | eval cluster_master=host | fields + site_pair cluster_master | fields - _* | dedup site_pair | mvexpand site_pair | dedup site_pair | rex field=site_pair "^(?<site_id>site\d+),\"?(?<indexer>.*)" | rex field=cluster_master "^(?<short_name_cm>[^\.]+)" | eval search="host=".indexer, host_count=1 | appendpipe [| stats values(indexer) as indexers by site_id short_name_cm | eval host_count=mvcount(indexers), search="host IN (".mvjoin(mvfilter(indexers!=""), ", ").")" | eval label=site_id." (".host_count." idxs @ ".short_name_cm ] | appendpipe [| stats values(indexer) as indexers dc(site_id) as site_count by short_name_cm | eval host_count=mvcount(indexers), search="host IN (".mvjoin(mvfilter(indexers!=""), ", ").")" | eval label=short_name_cm." (".host_count." idx ".site_count." sites)" ] | rex field=indexer "^(?<short_name_idx>[^\.]+)" | eval label=if(isnull(label), short_name_idx." (".site_id."@".short_name_cm.")", label) | stats max(host_count) as count by label search | sort 0 - count -24h@h now $selected_indexers$ None None -60m@m now if((round(relative_time(now(), $time.latest$)-relative_time(now(), $time.earliest$))/$time_resolution$)<31,31,round((relative_time(now(), $time.latest$)-relative_time(now(), $time.earliest$))/$time_resolution$)) Crude Low Medium High Ultra 500 if((round(relative_time(now(), $time.latest$)-relative_time(now(), $time.earliest$))/$time_resolution$)<31,31,round((relative_time(now(), $time.latest$)-relative_time(now(), $time.earliest$))/$time_resolution$))

Please select an cluster, site or host

 BucketReplicator full messages sending to indexers Indexer complaining The problem indexer indexer * | stats count by idx | sort - count | eval label=idx." (".count.")" * label idx | search BucketReplicator full idx=$selected_idx$ | chart limit=50 count by $selected_replication_indexer$ idx $click.name2$ Timeline of delays $selected_idx$ Complaining indexer Remote indexer Bucket ID indexer | xyseries _time $selected_timeline_attribute$ duration $row.bid$ $start$ $end$ Selected summary $selected_idx$ local, remote remote, local remote, bid local, bid remote local indexer host total duration blocked average duration blocked count sum(duration) | where _time > $selected_earliest$ and _time < $selected_latest$ | chart limit=80 $selected_summary_aggregator$ by $selected_summary_attribute$