]> ###v2024.02.18 - Fix sas-util to accomodate for controller/drivers designating the drives' paths as "scsi" not "sas" ###v2022.08.02 - Align return codes from "sdspin" (also per 6.11 changes) ###v2022.05.25 - Issue some more debug messages to syslog (vs. interactive "echo") when debug is set ###v2022.04.25 - Change version numbering to match common plugin wisdom - Minor code tidy-ups ###v0.86 2021-08-17 - With Unraid 6.10, some drives were spinning back up immediately. Reworked "IsSAS" logic to pre-detect drive types. - Streamline sdspin's exit code logic (basically aesthetics, no functional change in real world) ###v0.85 2021-02-06 - Failed to pull latest "exclusions" file in some installation scenarios. Thanks @xaositek. ###v0.84 2021-01-19 - Min Unraid version == 6.7.0 (previous didn't have req'd binaries) ###v0.83 2021-01-04 - Align copyright notices ###v0.82 2020-12-16 - In some configurations, SATA drives were mis-classified as SAS drives - In some cases, plugin removal did not reinstate Unraid's "sdspin" correctly ###v0.8 2020-12-09 - Adapt to Unraid's new spinup/down hook system (from 6.9.0-rc1 onwards). Plugin now supports both old and new systems - When spinning down off of rsyslog, send the sg_start command to the background so as not to block rsyslog if things get funny - Use sdparm rather than smartctl to detect if a drive is SAS - If smartctl version >= 7.2 or Unraid >= 6.9.0-rc1, do not install wrapper ###v0.71 2020-11-22 - Do not install if version > 6.9.0-beta35, per Tom's headsup ###v.07 2020-10-07 - Filter out syslog lines from some SAS devices rejecting ATA standby op (e0) issued by mdcmd - Consistent log messages' tag - Add some debug and testing tools - Implement an exclusion list. Initially for some SEAGATE and HITACHI SAS drives - When smartctl is evaded, give exit code 2 - as smartctl does with ATA. Thanks @segator - Adapt syslog include files to various Unraid configs :-( Reconfigure filter each time rsyslog.conf is modified. - Other fixes and improvements, major code reorg Thanks @SimonF for great feedback and suggesstions, thanks @jowe and others for testing efforts. ###v.06 2020-10-01 - Update support forum thread (new one, under Plugin Support) - Limit action to rotational drives (probably already covered by Unraid but making sure) - More code tidy-up ###v0.5 2020-09-26 - Pack as a plugin ###v.04 2020-09-22 - Some more cosmetic tidy-ups ###v0.3 2020-09-20 - Typo in message ###v.02 2020-09-20 - Fix a stupid packing bug - Some code tidy-up - Include version in messages - Log the installation Log () { echo "$@" logger -t "&name; plugin" "$@" } Log &name; v&version; installing which &prereqs; &> /dev/null || { Log "Error: At least one required binary not found on system." Log "Please make sure all of: &prereqs; - are installed." Log "You may want to try the nerdpack plugin for that." exit 1 } [ -f /usr/sbin/smartctl.wrapper ] && { Log "Removing leftover smartctl wrapper..." rm -f /usr/sbin/smartctl.wrapper } [ -f /etc/rsyslog.d/&syslogconf; ] && { Log "Removing leftover syslog conf file from previous installation..." rm -f "/etc/rsyslog.d/&syslogconf;" } [ -f "&sysloghook;" ] && { Log "Removing leftover syslog hook script from previous installation..." rm -f "&sysloghook;" } # Remove the syslog configurator if running in bg pkill sas-spindown rm -fr &plugindir;/* **Spin Down SAS Drives** A plugin to spin down idle SAS drives, according to Unraid configured spin-down schedule. This plugin assists the Unraid built-in disk spin down mechanism, by adding functionality supporting SAS drives (Unraid OS natively supports spinning down SATA drives). SAS drives are spun down by sending a standby_z, aka powermode 3, command. For Unraid version 6.9.0 and up, the built-in sdspin script is enhanced with SAS support. For Unraid versions up until 6.8.3, the plugin installs two main components: One is a syslog hook triggered whenever Unraid decides to spin down a drive; The second part is a wrapper for "smartctl", working around its deficiency of not supporting the "-n standby" flag for non-ATA drives (to be fixed in version 7.2). #!/bin/bash ####################################################### # Is this device on an exclusion list? # Bundled, v&version; # # ©right; IsExcluded () { ## return 1 #***** Temporary placeholder **** local DEV=${1#'/dev/'} local MODEL=$(echo $(cat /sys/block/$DEV/device/model)) # echo to trim local REV=$(echo $(cat /sys/block/$DEV/device/rev)) local VENDOR=$(echo $(cat /sys/block/$DEV/device/vendor)) local CTRLID=$(GetCtrlID $DEV) # PCI ID # This is the exception list. If it becomes larger - should improve search ## [[ "$MODEL" =~ ^ST[0-9]+NM[0-9]+3$ ]] || # Constellation ES.3 ## [[ "$VENDOR" == "SEAGATE" ]] || [[ "$MODEL" =~ ^HUC[0-9]+CSS600$ ]] || # Hitachi HUCnnnnCSS600 [[ "$MODEL" =~ X422_[A-Z0-9]{5}600A10 ]] || # Netapp rebrands of the Hitachi [[ "${MODEL}:${REV}" =~ MB8000JFECQ:HPD[4-7] ]] || # HPE MB8000JFECQ false # End the || chain } ####################################################### #!/bin/bash # Shared functions for SAS Spindown plugin # ©right; VERSION=&version; UNRAID_VERSION=$(cat /etc/unraid-version | cut -d '"' -f2) MDCMD=/usr/local/sbin/mdcmd SG_MAP=/usr/bin/sg_map SG_START=/usr/bin/sg_start SMARTCTL=/usr/sbin/smartctl REALSMART=/usr/sbin/smartctl*real SDPARM=/usr/sbin/sdparm HDPARM=/usr/sbin/hdparm LSPCI=/sbin/lspci DEBUG=&debug; [ -f &debugfile; ] && DEBUG=true . &plugindir;/drive_types 2> /dev/null # Trim enclosing whitespaces off a string Trim () { sed -e 's/^\s*//' -e 's/\s*$//' ; } Log () { logger -t "SAS Assist v$VERSION" -- "$@" ; } # Use PHP "version_compare" in bash Version_compare () { [[ $(php -r "echo version_compare(\"$1\", \"$2\", \"$3\") ? 'yes' : 'no' ;") == "yes" ]] } . &plugindir;/exclusions # Is this a SAS device? IsSAS () { if [ -n "${DRIVE_TYPE[${1#'/dev/'}]}" ] ; then # If we've already looked at this drive... [ "${DRIVE_TYPE[${1#'/dev/'}]}" == "SAS" ] # Return the rc from this comparison else trap "$(shopt -p nocasematch)" RETURN shopt -s nocasematch [[ $($SDPARM -ip di_target /dev/${1#'/dev/'} 2>&1 ) =~ transport:\ serial\ attached\ scsi ]] fi } # Is this a rotational (vs. solid state) device? IsRotational () { [ "$(cat /sys/block/${1#'/dev/'}/queue/rotational)" == 1 ] } # Is this SAS / ATA device spun down (standby_z)? IsSBY () { if IsSAS ${1#'/dev/'} ; then local OUTPUT=$($SDPARM -C sense /dev/${1#'/dev/'} 2>&1) [[ ${OUTPUT,,} =~ "standby condition activated" ]] else local OUTPUT=$($HDPARM -C /dev/${1#'/dev/'} 2>&1) [[ $? == 0 && ${OUTPUT,,} =~ "standby" && ! ${OUTPUT,,} =~ "bad/missing sense" ]] fi } # Get controller slot ID of an rdevice GetCtrlSlot () { readlink -m /sys/block/${1#'/dev/'} | sed -e 's=.*devices/pci[0-9a-f:]*/[0-9a-f:.]\+/[0-9a-f]*:==' -e 's=/.*==' } # Get controller PCI ID - primary and subsystem: 0000:0000:0000:0000 GetCtrlID () { ( cd $(readlink -m /sys/block/${1#'/dev/'} | sed 's=/host[0-9a-f].*==') # e.g. /sys/devices/pci0000:00/0000:00:17.0/0000:13:00.0 cat vendor device subsystem_vendor subsystem_device | paste -sd':' | sed 's/0x//g' ) } #!/bin/bash # # Deal with spin up/down status of HDDs # # This script is initiated from emhttpd, like so: # # sdspin <device> [up | down | status ] # # "device" is the HDD rdev name, such as "sdd". # # up == Spin the drive up # down == Spin the drive down # status == return the current status via rc # # Default (if no $2) is "status". # Exit code: # 0 - Success (if up/down), device spun up (if status) # 1 - Failure # 2 - Device spun down (if status) # Spin down/up SAS drives plugin # v&version; # # ©right; . &plugindir;/functions RDEVNAME=/dev/${1#'/dev/'} # So that we can be called with either "sdk" or "/dev/sdk" Hdparm () { OUTPUT=$($HDPARM $1 $RDEVNAME 2>&1) if [[ $? != 0 || ${OUTPUT,,} =~ "bad/missing sense" ]] ; then RC=1 fi $DEBUG && { Log "debug: $HDPARM $1 $RDEVNAME" Log "debug: $OUTPUT" ; } } RC=0 case ${2,,} in "up") if IsSAS $RDEVNAME ; then $DEBUG && Log "debug: $SG_START -rp1 $RDEVNAME" $SG_START -rp1 $RDEVNAME > /dev/null || RC=1 else Hdparm -S0 fi ;; "down") if IsSAS $RDEVNAME ; then if ! IsExcluded $RDEVNAME ; then if IsRotational $RDEVNAME ; then Log "Spinning down device $RDEVNAME" $DEBUG && Log "debug: $SG_START -rp3 $RDEVNAME" $SG_START -rp3 $RDEVNAME > /dev/null || RC=1 fi else Log "Device $RDEVNAME cannot be spun down - excluded" RC=1 fi else # Not SAS Hdparm -y fi ;; "status" | "") if IsSAS $RDEVNAME ; then OUTPUT=$($SDPARM -C sense $RDEVNAME 2>&1) if [[ $? != 0 ]] ; then RC=1 elif [[ ${OUTPUT,,} =~ "standby condition activated" ]] ; then RC=2 fi $DEBUG && { Log "debug: $SDPARM -C sense $RDEVNAME" Log "debug: $OUTPUT" ; } else Hdparm -C if [[ $RC == 0 && ${OUTPUT,,} =~ "standby" && ! ${OUTPUT,,} =~ "bad/missing sense" ]] ; then RC=2 fi fi ;; *) Log "Invalid op code $2" RC=1 ;; esac $DEBUG && Log "debug: exit $RC" exit $RC #!/bin/bash # # Spin down SAS drives plugin - config syslog # # This script will config the syslog hook for the spindown. It is needed because # vanilla Unraid and Dynamix create multiple flavors of rsyslog.conf and it is # not possible to have one include file that will address all situations. # This script is run by (a) plugin installation and (b) inotify hook, whenever # /etc/rsyslog.conf is changed. # # # v&version; # ©right; Main () { . &plugindir;/functions CFG=/etc/rsyslog.conf CFGDIR=/etc/rsyslog.d ME=$(readlink -f ${BASH_SOURCE[0]}) if [ "${1,,}" == "install" ] ; then ConfigSyslog echo "$ME" | at -M now else # Any time rsyslog.conf is accessed, do it again while inotifywait -q /etc/rsyslog.conf ; do ConfigSyslog done Log "inotity failed for rsyslog.conf - will not auto-configure syslog hook upon changes" fi } ConfigSyslog() { $DEBUG && Log "debug: syslog hook (re)config triggered" TMP=$(mktemp) if ! grep -qi '^$ruleset' $CFG ; then # No RuleSet in file; remove all ruleset indicators FILTER="sed -e ""/\$RuleSet/d"" -e ""/REMOTECONFIG/,999d""" elif grep -qi '^$ruleset remote' $CFG ; then # Remote RuleSet exists FILTER="cat" elif grep -qi '^$ruleset local' $CFG ; then # Remote RuleSet not active - leave only settings for "local" FILTER="sed -e ""/REMOTECONFIG/,999d""" else Log "Error: $CFG format unrecognized, will not install syslog hook." echo "Error: $CFG format unrecognized, will not install syslog hook." exit 2 fi cat &plugindir;/&syslogconf; | $FILTER > $TMP chmod 644 $TMP # If syslog config does not exist OR should be modified, update file and restart daemon if [ ! -f $CFGDIR/&syslogconf; ] || ! cmp -s $TMP $CFGDIR/&syslogconf; ; then Log "Configuring syslog filters at $CFGDIR/&syslogconf;" cp $TMP $CFGDIR/&syslogconf; sleep 2s /etc/rc.d/rc.rsyslogd restart 1>&2 fi rm -f $TMP } Main "$@" #!/bin/bash # # Spin down SAS drives - stopgap script until Unraid does organically. # # This script is initiated via syslog - when Unraid issues the "spindown n" message. # If the drive is SAS, the scipt will issue the commands to spin down a SAS drive. # # Spin up is not implemented - assumed to "just happen" when i/o is directed at drive. # # v&version; # ©right; . &plugindir;/functions $DEBUG && Log "debug: syslog filter triggered" grep -qe "mdcmd.*spindown" <<< "$1" || exit 0 # Get syslog line without line breaks and whatnot LINE=$(paste -sd ' ' <<< $1) # Obtain Unraid slot number being spun down, from syslog message SLOTNUM=$(sed -r 's/.*: *spindown ([[:digit:]]+).*/\1/' <<< $LINE) # Get the device name from the slot number RDEVNAME=$($MDCMD status | grep "rdevName.${SLOTNUM}=" | sed 's/.*=//') $DEBUG && Log "debug: syslog filter, R=$RDEVNAME S=$SLOTNUM" if IsSAS $RDEVNAME && IsRotational $RDEVNAME then if IsExcluded $RDEVNAME then Log "disk $SLOTNUM at /dev/$RDEVNAME not supported by SAS spindown plugin (excluded), not spun down" exit 0 fi # Figure out /dev/sgN type name from /dev/sdX name SGDEVNAME=$($SG_MAP | grep "/dev/$RDEVNAME\$" | awk '{print $1}' ) if [ "$SGDEVNAME" != "" ] ; then # Do the magic $DEBUG && Log "debug: $SG_START --readonly --pc=3 $SGDEVNAME" $SG_START --readonly --pc=3 $SGDEVNAME & Log "spinning down slot $SLOTNUM, device /dev/$RDEVNAME ($SGDEVNAME)" fi fi exit 0 #!/bin/bash # Spin down SAS drives - stopgap scripts until Unraid does organically. # # This script is a wrapper for "smartctl" - which in 7.1 does not support the "-n standby" # flag for SAS drive. This wrapper works around that, by checking whether the drive is SAS # and if so, avoid calling smartctl (return silently). # # v&version; # ©right; . &plugindir;/functions DEVICE="${@: -1}" $DEBUG && Log "debug: smartctl wrapper caller is $(cat /proc/$PPID/comm), grandpa is $(cat /proc/$(cat /proc/$PPID/stat | cut -d" " -f4)/comm), device $DEVICE, args \"$@\"" if [[ "$@" =~ \-n\ +standby\ || "$@" =~ "--nocheck=standby" ]] ; then if IsSBY $DEVICE ; then $DEBUG && Log "debug: device $DEVICE is spun down, smartctl evaded" echo "SAS Assist Plugin: $DEVICE is in standby mode (spun down), smartctl exits(2)" exit 2 # Match smartctl's exit code, thanks @segator fi fi $REALSMART "$@" # SAS Spindown Assist Plugin v&version; # ©right; $RuleSet local if $msg contains "spindown" and $.sasdone != 1 then { set $.sasdone = 1 ; ^&sysloghook; } # Some SAS devices spew errors when hit with ATA spindown opcode; filter out if re_match($msg, ".*error: mdcmd.*Input/output error \\(5\\): write.*") or re_match($msg, ".*md: do_drive_cmd:.*ATA_OP e[03] ioctl error.*") then stop #REMOTECONFIG do not move/remove this line $RuleSet remote if re_match($msg, ".*error: mdcmd.*Input/output error \\(5\\): write.*") or re_match($msg, ".*md: do_drive_cmd:.*ATA_OP e[03] ioctl error.*") then stop #!/bin/bash # Generate diagnostics for supporting the SAS spindown assist plugin. # ©right; VERSION=&version; DIAGFILE=/tmp/sas-spindown-diag.zip rm -f $DIAGFILE TDIR=$(mktemp -d) mdcmd status | sed -r 's/(^rdevid.*=|^diskid.*=).*/\1XXredactedXX/i' > $TDIR/mdcmd-status sg_map > $TDIR/sg_map lspci -vnnmm > $TDIR/lspci 2>/dev/null lspci -vvnn > $TDIR/lspcivv 2>/dev/null echo "SAS Spindown Diagnostic Tool v$VERSION" > $TDIR/version mdcmd status | grep -i "^rdevname." | sed -r 's/^rdevname\.([0-9]+)=(\S*)/\1 \2/i' | while read NUM RDEV ; do if [ -b /dev/$RDEV ] ; then echo '=================' echo "Slot $NUM device /dev/$RDEV" echo ls -lad /sys/block/$RDEV lspci -nns $(readlink -m /sys/block/$RDEV | sed -e 's=.*devices/pci[0-9a-f:]*/[0-9a-f:.]\+/[0-9a-f]*:==' -e 's=/.*==') echo '===' echo "smartctl -i /dev/$RDEV" smartctl -i /dev/$RDEV | sed -r 's/^(serial number:\s*).*/\1XXredactedXX/i' echo '===' echo "sdparm --command=sense /dev/$RDEV" sdparm --command=sense /dev/$RDEV echo '===' echo "sdparm --flexible -6 -v -S -p po /dev/$RDEV" sdparm --flexible -6 -v -S -p po /dev/$RDEV echo '===' echo "/sys/block/$RDEV/device/vendor=\"$(cat /sys/block/$RDEV/device/vendor)\"" echo "/sys/block/$RDEV/device/model=\"$(cat /sys/block/$RDEV/device/model)\"" fi done &> $TDIR/diag cp -p /etc/rsyslog.d/*.conf /etc/rsyslog.conf $TDIR/ cp -p /usr/sbin/smartctl $TDIR/ mkdir $TDIR/plugindir cp -p &plugindir;/* $TDIR/plugindir rsyslogd -nd &> $TDIR/rsyslogd-nd-out cd $TDIR zip -qr $DIAGFILE * rm -fr $TDIR echo "*** SAS-Spindown plugin diagnostics now available in $DIAGFILE." #!/bin/bash # # Build a list of drive types - SAS or otherwise. # # This script is executed upon plugin install (incl. reboot), and # when the array is started. # Spin down/up SAS drives plugin # v&version; # ©right; . &plugindir;/functions DRIVE_TYPES=&plugindir;/drive_types shopt -s nocasematch echo -e "#!/bin/bash\n\n# Automatically generated, $(date)\n\ndeclare -A DRIVE_TYPE" > $DRIVE_TYPES lsblk -So PATH,TYPE | grep -Po ".*(?= disk$)" | while read DISK ; do if [[ $($SDPARM -ip di_target $DISK 2>&1 ) =~ transport:\ serial\ attached\ scsi ]] ; then echo "DRIVE_TYPE[${DISK#'/dev/'}]=SAS" >> $DRIVE_TYPES else echo "DRIVE_TYPE[${DISK#'/dev/'}]=OTHER" >> $DRIVE_TYPES fi done #!/bin/bash # # This script is initiated from emhttpd, using the "Array Started" event upcall # Build a list of drive types - SAS or otherwise. # Spin down/up SAS drives plugin # v&version; # ©right; &plugindir;/build_drive_types . &plugindir;/functions Log () { echo "$@" logger -t "&name; plugin" "$@" } &debug; && exec 2>&1 &debug; && set -x if Version_compare "$UNRAID_VERSION" "6.9.0-beta41" ">" ; then NEW_FORMAT=true else NEW_FORMAT=false fi SMART_VER=$($SMARTCTL -V | head -n1 | cut -d' ' -f2) if Version_compare "$SMART_VER" "7.2" "<" ; then if [ ! -f /usr/local/sbin/smartctl_wrapper ] ; then Log "Installing a wrapper for smartctl $SMART_VER..." # Following is coded this way so that you can run this installer multiple times [ -f /usr/sbin/smartctl.real ] || cp -p /usr/sbin/smartctl /usr/sbin/smartctl.real [ -f /usr/sbin/smartctl.real ] && install &plugindir;/smartctl.wrapper -m 755 /usr/sbin/smartctl fi fi if $NEW_FORMAT ; then Log "Unraid version $UNRAID_VERSION detected, using post-6.9 method (emhttp hook)" Log "Installing SAS spindown/up script sdspin..." [ -f &emhttphookdir;/sdspin.unraid ] || mv &emhttphookdir;/sdspin &emhttphookdir;/sdspin.unraid install &plugindir;/sdspin -m 755 -Dt &emhttphookdir; else Log "Unraid version $UNRAID_VERSION detected, using legacy method (syslog hook)" Log "Installing syslog hook for SAS device spindown..." &plugindir;/sas-spindown-config-syslog install fi wget -q -t 3 -O &plugindir;/exclusions.latest &exclusionsURL; && { Log "Retrieved the latest exclusions file..." install &plugindir;/exclusions.latest -m 755 -Dt &pluginpermdir; } [ -f &pluginpermdir;/exclusions.latest ] && { Log "Installing most recent exclusions file..." mv &plugindir;/exclusions &plugindir;/exclusions.bundled install &pluginpermdir;/exclusions.latest -m 755 &plugindir;/exclusions } wget -q -t 3 -O &plugindir;/sas-util &utilURL; && { Log "Retrieved the latest sas-util..." install &plugindir;/sas-util -m 755 -Dt &pluginpermdir; } [ -f &pluginpermdir;/sas-util ] && { Log "Installing sas-util..." install &pluginpermdir;/sas-util -m 755 -Dt &plugindir; } Log "Populating drive types table..." &plugindir;/build_drive_types exit 0 . &plugindir;/functions Log () { echo "$@" logger -t "&name; plugin" "$@" } &debug; && exec 2>&1 &debug; && set -x if [ -f /usr/sbin/smartctl.real ] ; then Log "Removing the smartctl wrapper..." mv /usr/sbin/smartctl.real /usr/sbin/smartctl elif [ -f /usr/local/sbin/smartctl_wrapper.unraid ] ; then Log "Removing new smartctl wrapper..." mv /usr/local/sbin/smartctl_wrapper.unraid /usr/local/sbin/smartctl_wrapper fi if Version_compare "$UNRAID_VERSION" "6.9.0-beta41" ">" ; then NEW_FORMAT=true else NEW_FORMAT=false fi if $NEW_FORMAT ; then Log "Restoring Unraid OS spindown script..." mv &emhttphookdir;/sdspin.unraid &emhttphookdir;/sdspin else Log "Removing the syslog hook..." # Remove the syslog configurator if running in bg pkill sas-spindown sleep 2s rm -f "/etc/rsyslog.d/&syslogconf;" /etc/rc.d/rc.rsyslogd restart 1>&2 fi rm -fr "&plugindir;"