[Openais] corosync shutdown process

Andrew Beekhof andrew at beekhof.net
Wed Mar 9 00:10:09 PST 2011


Not enough information.
Create and attach a hb_report for the shutdown case.

On Tue, Mar 8, 2011 at 8:08 PM, Beau Sapach <beau.sapach at concordia.ab.ca> wrote:
> Hello everyone,
>
> I’ve got a 2-node cluster that exposes iSCSI targets backed by LVM volumes
> on top of a DRBD device.  For the most part I’ve got everything working as
> I’d like.  Manually moving resources works just fine, either using ‘move’ or
> by putting a node on standby.  Shutting down the corosync service on one
> node is another story though.  I have an order constraint in place to make
> iscsi-scst shutdown before stopping the LVM volume group but in the logs I
> see this:
>
> Mar 08 11:43:07 iscsitest2 crmd: [20755]: info: process_lrm_event: LRM
> operation clusterip_stop_0 (call=72, rc=0, cib-update=91, confirmed=true) ok
> Mar 08 11:43:07 iscsitest2 crmd: [20755]: info: do_lrm_rsc_op: Performing
> key=88:89:0:1effe13b-3093-4bf7-ae29-f764aaf22933 op=iscsi_target_stop_0 )
> Mar 08 11:43:07 iscsitest2 lrmd: [20752]: info: rsc:iscsi_target:73: stop
> Mar 08 11:43:07 iscsitest2 lrmd: [24805]: WARN: For LSB init script, no
> additional parameters are needed.
> Mar 08 11:43:07 iscsitest2 lrmd: [20752]: info: RA output:
> (iscsi_target:stop:stdout) Stopping iSCSI-SCST target service:
> Mar 08 11:43:07 iscsitest2 lrmd: [20752]: info: RA output:
> (iscsi_target:stop:stdout) succeeded.
> Mar 08 11:43:07 iscsitest2 lrmd: [20752]: info: RA output:
> (iscsi_target:stop:stdout) Removing iSCSI-SCST target modules:
> Mar 08 11:43:07 iscsitest2 crmd: [20755]: info: do_lrm_rsc_op: Performing
> key=51:89:0:1effe13b-3093-4bf7-ae29-f764aaf22933 op=drbd_lvm_stor:1_demote_0
> )
> Mar 08 11:43:07 iscsitest2 lrmd: [20752]: info: rsc:drbd_lvm_stor:1:74:
> demote
> Mar 08 11:43:07 iscsitest2 lrmd: [20752]: info: RA output:
> (drbd_lvm_stor:1:demote:stderr) 1: State change failed: (-12) Device is held
> open by someone
>
> The first line is fine, the iSCSI target IP should be shutdown first, then
> the target service and its modules are stopped/unloaded.  Next though I see
> corosync trying to demote the DRBD device that sits ‘under’ the LVM volume
> group, BEFORE it shuts down LVM… why are these things being done out of
> order?  Based on my constraint corosync should:
>
>
> Shutdown iscsi IP
> Shutdown iscsi-scst
> Shutdown LVM
> Demote drbd device
>
>
> The order constraint in my configuration looks like this:
>
> order san_startup inf: ms_drbd_lvm_stor:promote lvm_vg0 iscsi_target
> clusterip
>
> Lastly, I see, near the end of the log:
>
> Mar 08 11:43:11 iscsitest2 lrmd: [20752]: info: RA output:
> (drbd_lvm_stor:1:demote:stdout)
>
> Which, to me, looks like an incomplete line, followed by a number of attrd,
> crmd, stonithd & cib  ERROR messages indicating that the connection to the
> OpenAIS service has been lost.   I suppose this means that corosync doesn’t
> wait for proper resource migration before it shuts down which seems very
> strange to me.  Unless I’m missing something here, has anyone else run into
> anything like this?
>
> Beau
>
>
> _______________________________________________
> Openais mailing list
> Openais at lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/openais
>


More information about the Openais mailing list