Sep 5 11:28:14 localhost ansible-yum: Invoked with name=['http://albatros/bdc/drbd/rpms/20170905/drbd-utils-9.1.0-1.el7.centos.x86_64.rpm', 'http://albatros/bdc/drbd/rpms/20170905/drbd-pacemaker-9.1.0-1.el7.centos.x86_64.rpm', 'http://albatros/bdc/drbd/rpms/20170905/drbd-udev-9.1.0-1.el7.centos.x86_64.rpm', 'http://albatros/bdc/drbd/rpms/20170905/drbd-xen-9.1.0-1.el7.centos.x86_64.rpm', 'http://albatros/bdc/drbd/rpms/20170905/drbd-bash-completion-9.1.0-1.el7.centos.x86_64.rpm', 'http://albatros/bdc/drbd/rpms/20170905/drbd-9.1.0-1.el7.centos.x86_64.rpm', 'http://albatros/bdc/drbd/rpms/20170905/drbd-heartbeat-9.1.0-1.el7.centos.x86_64.rpm', 'http://albatros/bdc/drbd/rpms/20170905/kmod-drbd-9.0.9_3.10.0_327-1.el7.centos.x86_64.rpm', 'http://albatros/bdc/drbd/rpms/20170905/drbdmanage-0.99.10-1.noarch.rpm'] list=None install_repoquery=True conf_file=None disable_gpg_check=False state=present disablerepo=None update_cache=False enablerepo=None exclude=None validate_certs=True installroot=/ skip_broken=False Sep 5 11:28:23 localhost yum[5407]: Installed: drbd-utils-9.1.0-1.el7.centos.x86_64 Sep 5 11:28:23 localhost yum[5407]: Installed: drbd-pacemaker-9.1.0-1.el7.centos.x86_64 Sep 5 11:28:23 localhost yum[5407]: Installed: drbd-udev-9.1.0-1.el7.centos.x86_64 Sep 5 11:28:23 localhost yum[5407]: Installed: drbd-bash-completion-9.1.0-1.el7.centos.x86_64 Sep 5 11:28:23 localhost yum[5407]: Installed: drbd-xen-9.1.0-1.el7.centos.x86_64 Sep 5 11:28:23 localhost yum[5407]: Installed: drbdmanage-0.99.10-1.noarch Sep 5 11:28:23 localhost yum[5407]: Installed: drbd-9.1.0-1.el7.centos.x86_64 Sep 5 11:28:23 localhost yum[5407]: Installed: drbd-heartbeat-9.1.0-1.el7.centos.x86_64 Sep 5 11:28:54 localhost yum[5407]: Installed: kmod-drbd-9.0.9_3.10.0_327-1.el7.centos.x86_64 Sep 5 11:28:56 localhost ansible-command: Invoked with warn=True executable=None _uses_shell=False _raw_params=rpm -ivh --nodeps http://albatros/bdc/drbd/rpms/20170905/drbdmanage-docker-volume-0.7.1-1.noarch.rpm removes=None creates=None chdir=None Sep 5 11:28:56 localhost ansible-blockinfile: Invoked with directory_mode=None force=None remote_src=None insertafter=None path=/etc/drbdmanaged.cfg owner=None follow=False marker=# {mark} ANSIBLE MANAGED BLOCK group=None insertbefore=None unsafe_writes=None create=True state=present content=None serole=None setype=None selevel=None regexp=None validate=None src=None seuser=None delimiter=None mode=None attributes=None backup=False block=[LOCAL]#012drbdctrl-vg = vg00 Sep 5 11:28:57 localhost ansible-systemd: Invoked with no_block=False name=docker-drbdmanage-plugin enabled=True daemon_reload=False state=started user=False masked=None Sep 5 11:28:57 localhost systemd: Listening on DRBDmanage Docker Volume Plugin. Sep 5 11:28:57 localhost systemd: Starting DRBDmanage Docker Volume Plugin. Sep 5 11:28:57 localhost systemd: Started DRBDmanage Docker Volume Plugin. Sep 5 11:28:57 localhost systemd: Starting DRBDmanage Docker Volume Plugin... Sep 5 11:28:57 localhost dbus[548]: [system] Activating service name='org.drbd.drbdmanaged' (using servicehelper) Sep 5 11:28:57 localhost dbus-daemon: dbus[548]: [system] Activating service name='org.drbd.drbdmanaged' (using servicehelper) Sep 5 11:28:57 localhost drbdmanaged[14194]: INFO DRBDmanage server, version 0.99.10 -- initializing on node 'node-10-9-4-216' Sep 5 11:28:57 localhost dbus[548]: [system] Successfully activated service 'org.drbd.drbdmanaged' Sep 5 11:28:57 localhost drbdmanaged[14194]: INFO DrbdAdm: Running external command: drbdadm -vvv adjust .drbdctrl Sep 5 11:28:57 localhost dbus-daemon: dbus[548]: [system] Successfully activated service 'org.drbd.drbdmanaged' Sep 5 11:28:57 localhost drbdmanaged[14194]: ERROR DrbdAdm: External command 'drbdadm': Exit code 1 Sep 5 11:28:57 localhost kernel: drbd: module verification failed: signature and/or required key missing - tainting kernel Sep 5 11:28:57 localhost kernel: drbd: initialized. Version: 9.0.9-1 (api:2/proto:86-112) Sep 5 11:28:57 localhost kernel: drbd: GIT-hash: f7b979e7af01813e031aac579140237640c94569 build by packer@localhost.localdomain, 2017-09-05 10:31:29 Sep 5 11:28:57 localhost kernel: drbd: registered as block device major 147 Sep 5 11:28:57 localhost dbus-daemon: .drbdctrl: No such resource Sep 5 11:28:57 localhost dbus-daemon: # No currently configured DRBD found. Sep 5 11:28:57 localhost drbdmanaged[14194]: INFO DRBDManage starting as satellite node Sep 5 11:28:57 localhost drbdmanaged[14194]: ERROR drbdsetup events tracing has failed, restarting Sep 5 11:28:57 localhost drbdmanaged[14194]: INFO drbdsetup events tracing reestablished Sep 5 11:28:57 localhost drbdmanaged[14194]: INFO Reading initial DRBD control volume status Sep 5 11:28:57 localhost drbdmanaged[14194]: INFO Finished reading initial DRBD control volume status Sep 5 11:28:57 localhost ansible-command: Invoked with warn=True executable=None _uses_shell=False _raw_params=drbdmanage init --quiet 10.9.4.216 removes=None creates=None chdir=None Sep 5 11:28:58 localhost kernel: drbd .drbdctrl: Starting worker thread (from drbdsetup [14305]) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: disk( Diskless -> Attaching ) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: Maximum number of peer devices = 31 Sep 5 11:28:58 localhost kernel: drbd .drbdctrl: Method to ensure write ordering: flush Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: Adjusting my ra_pages to backing device's (32 -> 1024) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: drbd_bm_resize called with capacity == 8112 Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: resync bitmap: bits=1014 words=496 pages=1 Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: recounting of set bits took additional 0ms Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: disk( Attaching -> Inconsistent ) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: attached to current UUID: 0000000000000004 Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: disk( Diskless -> Attaching ) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: Maximum number of peer devices = 31 Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: Adjusting my ra_pages to backing device's (32 -> 1024) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: drbd_bm_resize called with capacity == 8112 Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: resync bitmap: bits=1014 words=496 pages=1 Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: recounting of set bits took additional 0ms Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: disk( Attaching -> Inconsistent ) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: attached to current UUID: 0000000000000004 Sep 5 11:28:58 localhost kernel: drbd .drbdctrl: Preparing cluster-wide state change 3185125642 (0->-1 7683/4609) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl: Committing cluster-wide state change 3185125642 (0ms) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl: role( Secondary -> Primary ) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: disk( Inconsistent -> UpToDate ) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: disk( Inconsistent -> UpToDate ) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: size = 4056 KB (4056 KB) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: size = 4056 KB (4056 KB) Sep 5 11:28:58 localhost kernel: drbd .drbdctrl: Forced to consider local data as UpToDate! Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/0 drbd0: new current UUID: 48F9E96378C0E133 weak: FFFFFFFFFFFFFFFE Sep 5 11:28:58 localhost kernel: drbd .drbdctrl/1 drbd1: new current UUID: 90E9BF45847AFEA7 weak: FFFFFFFFFFFFFFFE Sep 5 11:28:58 localhost kernel: drbd .drbdctrl: role( Primary -> Secondary ) Sep 5 11:28:58 localhost drbdmanaged[14194]: INFO server shutdown requested through D-Bus Sep 5 11:28:58 localhost drbdmanaged[14194]: INFO server shutdown (requested by shutdown API call) Sep 5 11:28:58 localhost drbdmanaged[14194]: INFO shutting down satellites Sep 5 11:28:58 localhost drbdmanaged[14194]: INFO shutting down the TCPServer Sep 5 11:28:58 localhost drbdmanaged[14194]: INFO shutting down the control volume Sep 5 11:28:58 localhost drbdmanaged[14194]: INFO DrbdAdm: Running external command: drbdadm -vvv down .drbdctrl Sep 5 11:28:58 localhost drbdmanaged[14194]: ERROR DrbdAdm: External command 'drbdadm': Exit code 1 Sep 5 11:28:58 localhost drbdmanaged[14194]: ERROR drbdadm/stderr: no resources defined! Sep 5 11:28:58 localhost drbdmanaged[14194]: INFO shutting down DRBD events processing Sep 5 11:28:58 localhost drbdmanaged[14194]: INFO server shutdown complete, exiting Sep 5 11:28:58 localhost dbus[548]: [system] Activating service name='org.drbd.drbdmanaged' (using servicehelper) Sep 5 11:28:58 localhost dbus-daemon: dbus[548]: [system] Activating service name='org.drbd.drbdmanaged' (using servicehelper) Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO DRBDmanage server, version 0.99.10 -- initializing on node 'node-10-9-4-216' Sep 5 11:28:59 localhost dbus[548]: [system] Successfully activated service 'org.drbd.drbdmanaged' Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO DrbdAdm: Running external command: drbdadm -vvv adjust .drbdctrl Sep 5 11:28:59 localhost dbus-daemon: dbus[548]: [system] Successfully activated service 'org.drbd.drbdmanaged' Sep 5 11:28:59 localhost drbdmanaged[14335]: ERROR DrbdAdm: External command 'drbdadm': Exit code 1 Sep 5 11:28:59 localhost dbus-daemon: .drbdctrl role:Secondary Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO DRBDManage starting as potential leader node Sep 5 11:28:59 localhost drbdmanaged[14335]: ERROR drbdsetup events tracing has failed, restarting Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO drbdsetup events tracing reestablished Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO Reading initial DRBD control volume status Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO Finished reading initial DRBD control volume status Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO Leader election by wait for connections Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO DrbdAdm: Running external command: drbdsetup wait-connect-resource --wait-after-sb=yes --wfc-timeout=2 .drbdctrl Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO DrbdAdm: External command 'drbdsetup': Exit code 0 Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO DrbdAdm: Running external command: drbdsetup primary .drbdctrl Sep 5 11:28:59 localhost kernel: drbd .drbdctrl: Preparing cluster-wide state change 2208040552 (0->-1 3/1) Sep 5 11:28:59 localhost kernel: drbd .drbdctrl: Committing cluster-wide state change 2208040552 (1ms) Sep 5 11:28:59 localhost kernel: drbd .drbdctrl: role( Secondary -> Primary ) Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO DrbdAdm: External command 'drbdsetup': Exit code 0 Sep 5 11:28:59 localhost drbdmanaged[14335]: INFO Leader election: won election Sep 5 11:28:59 localhost kernel: drbd .drbdctrl/1 drbd1: new current UUID: D0FDD87839B79CF5 weak: FFFFFFFFFFFFFFFE Sep 5 11:28:59 localhost python2: File "/usr/libexec/docker/drbdmanage-docker-volume", line 81, in Sep 5 11:28:59 localhost python2: File "/usr/libexec/docker/drbdmanage-docker-volume", line 83, in DockerHandler Sep 5 11:28:59 localhost python2: dm = DrbdManageDriver() Sep 5 11:28:59 localhost python2: File "/usr/libexec/docker/drbdmanage-docker-volume", line 51, in __init__ Sep 5 11:28:59 localhost python2: super(DrbdManageDriver, self).__init__() Sep 5 11:28:59 localhost python2: File "/usr/lib/python2.7/site-packages/drbdmanage/clienthelper.py", line 58, in __init__ Sep 5 11:28:59 localhost python2: File "/usr/lib/python2.7/site-packages/drbdmanage/clienthelper.py", line 84, in dbus_connect Sep 5 11:28:59 localhost systemd: docker-drbdmanage-plugin.service: main process exited, code=exited, status=1/FAILURE Sep 5 11:28:59 localhost systemd: Unit docker-drbdmanage-plugin.service entered failed state. Sep 5 11:28:59 localhost systemd: docker-drbdmanage-plugin.service failed. Sep 5 11:29:01 localhost drbdmanaged[14335]: INFO DrbdAdm: Running external command: drbdadm -vvv adjust .drbdctrl Sep 5 11:29:01 localhost drbdmanaged[14335]: INFO DrbdAdm: External command 'drbdadm': Exit code 0 Sep 5 11:29:01 localhost ansible-command: Invoked with warn=True executable=None _uses_shell=False _raw_params=drbdmanage add-node -j node-10-9-4-231 10.9.4.231 removes=None creates=None chdir=None Sep 5 11:29:01 localhost ansible-command: Invoked with warn=True executable=None _uses_shell=False _raw_params=drbdmanage add-node -j node-10-9-4-166 10.9.4.166 removes=None creates=None chdir=None Sep 5 11:29:01 localhost drbdmanaged[14335]: INFO DrbdAdm: Running external command: drbdadm -vvv adjust .drbdctrl Sep 5 11:29:01 localhost kernel: drbd .drbdctrl node-10-9-4-166: Starting sender thread (from drbdsetup [14493]) Sep 5 11:29:01 localhost drbdmanaged[14335]: INFO DrbdAdm: External command 'drbdadm': Exit code 0 Sep 5 11:29:01 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( StandAlone -> Unconnected ) Sep 5 11:29:01 localhost kernel: drbd .drbdctrl node-10-9-4-166: Starting receiver thread (from drbd_w_.drbdctr [14306]) Sep 5 11:29:01 localhost kernel: drbd .drbdctrl/0 drbd0: new current UUID: 2BEFE165D2E607A5 weak: FFFFFFFFFFFFFFFE Sep 5 11:29:01 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Unconnected -> Connecting ) Sep 5 11:29:01 localhost drbdmanaged[14335]: INFO DrbdAdm: Running external command: drbdadm -vvv adjust .drbdctrl Sep 5 11:29:01 localhost kernel: drbd .drbdctrl node-10-9-4-231: Starting sender thread (from drbdsetup [14504]) Sep 5 11:29:01 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( StandAlone -> Unconnected ) Sep 5 11:29:01 localhost drbdmanaged[14335]: INFO DrbdAdm: External command 'drbdadm': Exit code 0 Sep 5 11:29:01 localhost kernel: drbd .drbdctrl node-10-9-4-231: Starting receiver thread (from drbd_w_.drbdctr [14306]) Sep 5 11:29:01 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Unconnected -> Connecting ) Sep 5 11:29:02 localhost kernel: drbd .drbdctrl node-10-9-4-231: Handshake to peer 2 successful: Agreed network protocol version 112 Sep 5 11:29:02 localhost kernel: drbd .drbdctrl node-10-9-4-231: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. Sep 5 11:29:02 localhost kernel: drbd .drbdctrl node-10-9-4-231: Peer authenticated using 32 bytes HMAC Sep 5 11:29:02 localhost kernel: drbd .drbdctrl node-10-9-4-231: Starting ack_recv thread (from drbd_r_.drbdctr [14508]) Sep 5 11:29:02 localhost kernel: drbd .drbdctrl node-10-9-4-166: Handshake to peer 1 successful: Agreed network protocol version 112 Sep 5 11:29:02 localhost kernel: drbd .drbdctrl node-10-9-4-166: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. Sep 5 11:29:02 localhost kernel: drbd .drbdctrl node-10-9-4-166: Peer authenticated using 32 bytes HMAC Sep 5 11:29:02 localhost kernel: drbd .drbdctrl node-10-9-4-166: Starting ack_recv thread (from drbd_r_.drbdctr [14499]) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Preparing cluster-wide state change 3785290307 (0->1 499/145) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Declined by peer node-10-9-4-166 (id: 1), see the kernel log there Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Aborting cluster-wide state change 3785290307 (0ms) rv = -10 Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Failure to connect; retrying Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Connecting -> NetworkFailure ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: ack_receiver terminated Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Terminating ack_recv thread Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Connection closed Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( NetworkFailure -> Unconnected ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Restarting receiver thread Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Unconnected -> Connecting ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Preparing cluster-wide state change 474452910 (0->2 499/145) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Declined by peer node-10-9-4-231 (id: 2), see the kernel log there Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Aborting cluster-wide state change 474452910 (1ms) rv = -10 Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Failure to connect; retrying Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Connecting -> NetworkFailure ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: ack_receiver terminated Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Terminating ack_recv thread Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Connection closed Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( NetworkFailure -> Unconnected ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Restarting receiver thread Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Unconnected -> Connecting ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Handshake to peer 1 successful: Agreed network protocol version 112 Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Peer authenticated using 32 bytes HMAC Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Starting ack_recv thread (from drbd_r_.drbdctr [14499]) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Handshake to peer 2 successful: Agreed network protocol version 112 Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Peer authenticated using 32 bytes HMAC Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Starting ack_recv thread (from drbd_r_.drbdctr [14508]) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Preparing cluster-wide state change 255890997 (0->1 499/145) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Declined by peer node-10-9-4-166 (id: 1), see the kernel log there Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Aborting cluster-wide state change 255890997 (1ms) rv = -10 Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Failure to connect; retrying Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Connecting -> NetworkFailure ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: ack_receiver terminated Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Terminating ack_recv thread Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Connection closed Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( NetworkFailure -> Unconnected ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: Restarting receiver thread Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Unconnected -> Connecting ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Preparing cluster-wide state change 11808752 (0->2 499/145) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Declined by peer node-10-9-4-231 (id: 2), see the kernel log there Sep 5 11:29:03 localhost kernel: drbd .drbdctrl: Aborting cluster-wide state change 11808752 (0ms) rv = -10 Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Failure to connect; retrying Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Connecting -> NetworkFailure ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: ack_receiver terminated Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Terminating ack_recv thread Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Connection closed Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( NetworkFailure -> Unconnected ) Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: Restarting receiver thread Sep 5 11:29:03 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Unconnected -> Connecting ) Sep 5 11:29:04 localhost drbdmanaged[14335]: INFO updating storage pool information Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Handshake to peer 1 successful: Agreed network protocol version 112 Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Peer authenticated using 32 bytes HMAC Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Starting ack_recv thread (from drbd_r_.drbdctr [14499]) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: Handshake to peer 2 successful: Agreed network protocol version 112 Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: Peer authenticated using 32 bytes HMAC Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: Starting ack_recv thread (from drbd_r_.drbdctr [14508]) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl: Preparing cluster-wide state change 4185636038 (0->1 499/145) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl: Declined by peer node-10-9-4-166 (id: 1), see the kernel log there Sep 5 11:29:04 localhost kernel: drbd .drbdctrl: Aborting cluster-wide state change 4185636038 (0ms) rv = -10 Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Failure to connect; retrying Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Connecting -> NetworkFailure ) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: ack_receiver terminated Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Terminating ack_recv thread Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Connection closed Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( NetworkFailure -> Unconnected ) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Restarting receiver thread Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Unconnected -> Connecting ) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl: Preparing cluster-wide state change 4044658125 (0->2 499/145) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl: Declined by peer node-10-9-4-231 (id: 2), see the kernel log there Sep 5 11:29:04 localhost kernel: drbd .drbdctrl: Aborting cluster-wide state change 4044658125 (0ms) rv = -10 Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: Failure to connect; retrying Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Connecting -> NetworkFailure ) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: ack_receiver terminated Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: Terminating ack_recv thread Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: Connection closed Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( NetworkFailure -> Unconnected ) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: Restarting receiver thread Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Unconnected -> Connecting ) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Handshake to peer 1 successful: Agreed network protocol version 112 Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Authentication of peer failed Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Connecting -> Disconnecting ) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Connection closed Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: conn( Disconnecting -> StandAlone ) Sep 5 11:29:04 localhost kernel: drbd .drbdctrl node-10-9-4-166: Terminating receiver thread Sep 5 11:29:05 localhost kernel: drbd .drbdctrl node-10-9-4-231: Handshake to peer 2 successful: Agreed network protocol version 112 Sep 5 11:29:05 localhost kernel: drbd .drbdctrl node-10-9-4-231: Feature flags enabled on protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME. Sep 5 11:29:05 localhost kernel: drbd .drbdctrl node-10-9-4-231: Authentication of peer failed Sep 5 11:29:05 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Connecting -> Disconnecting ) Sep 5 11:29:05 localhost kernel: drbd .drbdctrl node-10-9-4-231: Connection closed Sep 5 11:29:05 localhost kernel: drbd .drbdctrl node-10-9-4-231: conn( Disconnecting -> StandAlone ) Sep 5 11:29:05 localhost kernel: drbd .drbdctrl node-10-9-4-231: Terminating receiver thread