[Openais] Trunk broken?

Steven Dake sdake at redhat.com
Tue Apr 25 08:24:26 PDT 2006


Fabien,

It appears one of your nodes is dropping the token.  I have not made any
changes to the token sending routines that I can recall.  Are you still
using the "netmtu" command to reduce the size of the network mtu?

Try an svn update to get all the latest bits.  If that doesn't work, if
you can identify the revision that breaks the code for you, that would
really help.

(It works here with 4 nodes).

Regards
-steve


On Tue, 2006-04-25 at 15:27 +0200, Fabien THOMAS wrote:
> i've updated to latest trunk and cannot get it to works with two  
> nodes it loops :
> 
> Apr 25 13:15:05.975526 [MAIN ] Could not lock memory of service to  
> avoid page faults
> Apr 25 13:15:05.976208 [TOTEM] Token Timeout (1000 ms) retransmit  
> timeout (238 ms)
> Apr 25 13:15:05.976322 [TOTEM] token hold (180 ms) retransmits before  
> loss (4 retrans)
> Apr 25 13:15:05.976558 [TOTEM] join (100 ms) consensus (200 ms) merge  
> (200 ms)
> Apr 25 13:15:05.976632 [TOTEM] downcheck (1000 ms) fail to recv const  
> (50 msgs)
> Apr 25 13:15:05.976704 [TOTEM] seqno unchanged const (30 rotations)  
> Maximum network MTU 1500
> Apr 25 13:15:05.976784 [TOTEM] window size per rotation (50 messages)  
> maximum messages per rotation (17 messages)
> Apr 25 13:15:05.977016 [TOTEM] send threads (0 threads)
> Apr 25 13:15:05.977084 [TOTEM] heartbeat_failures_allowed (0)
> Apr 25 13:15:05.977151 [TOTEM] max_network_delay (50 ms)
> Apr 25 13:15:05.977831 [TOTEM] HeartBeat is Disabled. To enable set  
> heartbeat_failures_allowed > 0
> Apr 25 13:15:05.981252 [TOTEM] Receive multicast socket recv buffer  
> size (144000 bytes).
> Apr 25 13:15:05.981616 [TOTEM] Transmit multicast socket send buffer  
> size (144000 bytes).
> Apr 25 13:15:05.982116 [TOTEM] The network interface [10.2.1.7] is  
> now up.
> Apr 25 13:15:05.982346 [TOTEM] Created or loaded sequence id  
> 63728.10.2.1.7 for this ring.
> Apr 25 13:15:05.983214 [TOTEM] entering GATHER state.
> Apr 25 13:15:05.983694 [SERV ] Initialising service handler 'openais  
> cluster membership service B.01.01'
> Apr 25 13:15:05.984103 [SERV ] Initialising service handler 'openais  
> availability management framework B.01.01'
> Apr 25 13:15:05.984221 [SERV ] Initialising service handler 'openais  
> checkpoint service B.01.01'
> Apr 25 13:15:05.984308 [SERV ] Initialising service handler 'openais  
> event service B.01.01'
> Apr 25 13:15:05.984578 [SERV ] Initialising service handler 'openais  
> distributed locking service B.01.01'
> Apr 25 13:15:05.984672 [SERV ] Initialising service handler 'openais  
> message service B.01.01'
> Apr 25 13:15:05.984757 [SERV ] Initialising service handler 'openais  
> configuration service'
> Apr 25 13:15:05.984841 [SERV ] Initialising service handler 'openais  
> cluster closed process group service v1.01'
> Apr 25 13:15:05.987150 [MAIN ] AIS Executive Service: started and  
> ready to provide service.
> Apr 25 13:15:05.988130 [TOTEM] Creating commit token because I am the  
> rep.
> Apr 25 13:15:05.988277 [TOTEM] Saving state aru 0 high seq received 0
> Apr 25 13:15:05.989269 [TOTEM] Storing new sequence id for ring 63732
> Apr 25 13:15:05.989792 [TOTEM] entering COMMIT state.
> Apr 25 13:15:05.991214 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:05.992591 [TOTEM] position [0] member 10.2.1.7:
> Apr 25 13:15:05.992747 [TOTEM] previous ring seq 63728 rep 10.2.1.7
> Apr 25 13:15:05.992821 [TOTEM] aru 0 high delivered 0 received flag 0
> Apr 25 13:15:05.993051 [TOTEM] Did not need to originate any messages  
> in recovery.
> Apr 25 13:15:05.993661 [TOTEM] Sending initial ORF token
> Apr 25 13:15:05.999589 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:05.999746 [CLM  ] New Configuration:
> Apr 25 13:15:05.999813 [CLM  ] Members Left:
> Apr 25 13:15:06.000289 [CLM  ] Members Joined:
> Apr 25 13:15:06.001124 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:06.001230 [CLM  ] New Configuration:
> Apr 25 13:15:06.001304 [CLM  ] 	10.2.1.7
> Apr 25 13:15:06.001525 [CLM  ] Members Left:
> Apr 25 13:15:06.001591 [CLM  ] Members Joined:
> Apr 25 13:15:06.001663 [CLM  ] 	10.2.1.7
> Apr 25 13:15:06.001776 [SYNC ] This node is within the non-primary  
> component and will NOT provide any services.
> Apr 25 13:15:06.064051 [TOTEM] entering OPERATIONAL state.
> Apr 25 13:15:06.075652 [YKD  ] This processor is within the primary  
> component.
> Apr 25 13:15:06.076080 [SYNC ] This node is within the primary  
> component and will provide service.
> Apr 25 13:15:06.077375 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:06.077681 [SYNC ] Synchronization actions starting for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:06.077831 [SYNC ] Synchronization actions done for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:06.079122 [CLM  ] got nodejoin message 10.2.1.7
> Apr 25 13:15:06.080331 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:06.080675 [SYNC ] Synchronization actions starting for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:06.080805 [SYNC ] Synchronization actions done for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:06.082876 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:06.083162 [SYNC ] Synchronization actions starting for  
> (openais event service B.01.01)
> Apr 25 13:15:06.087210 [SYNC ] Synchronization actions done for  
> (openais event service B.01.01)
> Apr 25 13:15:06.088651 [TOTEM] entering GATHER state.
> Apr 25 13:15:06.092611 [TOTEM] Saving state aru 2a high seq received 2a
> Apr 25 13:15:06.093620 [TOTEM] Storing new sequence id for ring 63736
> Apr 25 13:15:06.093816 [TOTEM] entering COMMIT state.
> Apr 25 13:15:06.097226 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:06.098654 [TOTEM] position [0] member 10.2.1.6:
> Apr 25 13:15:06.098812 [TOTEM] previous ring seq 63728 rep 10.2.1.6
> Apr 25 13:15:06.098886 [TOTEM] aru 2a high delivered 2a received flag 0
> Apr 25 13:15:06.099119 [TOTEM] position [1] member 10.2.1.7:
> Apr 25 13:15:06.099197 [TOTEM] previous ring seq 63732 rep 10.2.1.7
> Apr 25 13:15:06.099268 [TOTEM] aru 2a high delivered 2a received flag 0
> Apr 25 13:15:06.099347 [TOTEM] copying all old ring messages from 2b-2a.
> Apr 25 13:15:06.099424 [TOTEM] Originated 0 messages in RECOVERY.
> Apr 25 13:15:06.099667 [TOTEM] Originated for recovery:
> Apr 25 13:15:06.099736 [TOTEM] Not Originated for recovery:
> Apr 25 13:15:06.117182 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:06.117377 [CLM  ] New Configuration:
> Apr 25 13:15:06.117621 [CLM  ] 	10.2.1.7
> Apr 25 13:15:06.117691 [CLM  ] Members Left:
> Apr 25 13:15:06.117756 [CLM  ] Members Joined:
> Apr 25 13:15:06.117938 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:06.118169 [CLM  ] New Configuration:
> Apr 25 13:15:06.118243 [CLM  ] 	10.2.1.6
> Apr 25 13:15:06.118315 [CLM  ] 	10.2.1.7
> Apr 25 13:15:06.118381 [CLM  ] Members Left:
> Apr 25 13:15:06.118596 [CLM  ] Members Joined:
> Apr 25 13:15:06.118671 [CLM  ] 	10.2.1.6
> Apr 25 13:15:06.118763 [SYNC ] This node is within the non-primary  
> component and will NOT provide any services.
> Apr 25 13:15:06.170331 [TOTEM] entering OPERATIONAL state.
> Apr 25 13:15:07.172328 [TOTEM] The token was lost in state 1 from  
> timer 83ce000
> Apr 25 13:15:07.174080 [TOTEM] Receive multicast socket recv buffer  
> size (144000 bytes).
> Apr 25 13:15:07.174418 [TOTEM] Transmit multicast socket send buffer  
> size (144000 bytes).
> Apr 25 13:15:07.175429 [TOTEM] entering GATHER state.
> Apr 25 13:15:07.382250 [TOTEM] entering GATHER state.
> Apr 25 13:15:07.406583 [TOTEM] Creating commit token because I am the  
> rep.
> Apr 25 13:15:07.406788 [TOTEM] Saving state aru 10 high seq received 10
> Apr 25 13:15:07.407777 [TOTEM] Storing new sequence id for ring 63744
> Apr 25 13:15:07.408128 [TOTEM] entering COMMIT state.
> Apr 25 13:15:07.408720 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:07.409974 [TOTEM] position [0] member 10.2.1.7:
> Apr 25 13:15:07.410128 [TOTEM] previous ring seq 63736 rep 10.2.1.6
> Apr 25 13:15:07.410203 [TOTEM] aru 10 high delivered 0 received flag 0
> Apr 25 13:15:07.410285 [TOTEM] copying all old ring messages from 11-10.
> Apr 25 13:15:07.410520 [TOTEM] Originated 0 messages in RECOVERY.
> Apr 25 13:15:07.410593 [TOTEM] Originated for recovery:
> Apr 25 13:15:07.410659 [TOTEM] Not Originated for recovery:
> Apr 25 13:15:07.411246 [TOTEM] Sending initial ORF token
> Apr 25 13:15:07.418268 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:07.418654 [CLM  ] New Configuration:
> Apr 25 13:15:07.418734 [CLM  ] 	10.2.1.7
> Apr 25 13:15:07.418804 [CLM  ] Members Left:
> Apr 25 13:15:07.419025 [CLM  ] 	10.2.1.6
> Apr 25 13:15:07.419092 [CLM  ] Members Joined:
> Apr 25 13:15:07.419181 [CKPT ] clean_checkpoint_list: List is empty
> Apr 25 13:15:07.419280 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:07.419502 [CLM  ] New Configuration:
> Apr 25 13:15:07.419577 [CLM  ] 	10.2.1.7
> Apr 25 13:15:07.419644 [CLM  ] Members Left:
> Apr 25 13:15:07.419708 [CLM  ] Members Joined:
> Apr 25 13:15:07.419794 [SYNC ] This node is within the non-primary  
> component and will NOT provide any services.
> Apr 25 13:15:07.471209 [TOTEM] entering OPERATIONAL state.
> Apr 25 13:15:07.482646 [YKD  ] This processor is within the primary  
> component.
> Apr 25 13:15:07.483046 [SYNC ] This node is within the primary  
> component and will provide service.
> Apr 25 13:15:07.483189 [YKD  ] This processor is within the primary  
> component.
> Apr 25 13:15:07.483264 [SYNC ] This node is within the primary  
> component and will provide service.
> Apr 25 13:15:07.484664 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:07.484811 [SYNC ] Synchronization actions starting for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:07.485107 [SYNC ] Synchronization actions done for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:07.486871 [CLM  ] got nodejoin message 10.2.1.7
> Apr 25 13:15:07.488347 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:07.488651 [SYNC ] Synchronization actions starting for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:07.488777 [SYNC ] Synchronization actions done for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:07.490859 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:07.491146 [SYNC ] Synchronization actions starting for  
> (openais event service B.01.01)
> Apr 25 13:15:07.491245 [MAIN ] Can't find cluster node at 10.2.1.6
> Apr 25 13:15:07.495344 [SYNC ] Synchronization actions done for  
> (openais event service B.01.01)
> Apr 25 13:15:08.368732 [TOTEM] entering GATHER state.
> Apr 25 13:15:08.371716 [TOTEM] Saving state aru 3e high seq received 3e
> Apr 25 13:15:08.372794 [TOTEM] Storing new sequence id for ring 63748
> Apr 25 13:15:08.373171 [TOTEM] entering COMMIT state.
> Apr 25 13:15:08.375755 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:08.379433 [TOTEM] position [0] member 10.2.1.6:
> Apr 25 13:15:08.379870 [TOTEM] previous ring seq 63736 rep 10.2.1.6
> Apr 25 13:15:08.379946 [TOTEM] aru 0 high delivered 0 received flag 0
> Apr 25 13:15:08.380232 [TOTEM] position [1] member 10.2.1.7:
> Apr 25 13:15:08.380311 [TOTEM] previous ring seq 63744 rep 10.2.1.7
> Apr 25 13:15:08.380383 [TOTEM] aru 3e high delivered 3e received flag 0
> Apr 25 13:15:08.380467 [TOTEM] copying all old ring messages from 3f-3e.
> Apr 25 13:15:08.381226 [TOTEM] Originated 0 messages in RECOVERY.
> Apr 25 13:15:08.381323 [TOTEM] Originated for recovery:
> Apr 25 13:15:08.381391 [TOTEM] Not Originated for recovery:
> Apr 25 13:15:08.396649 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:08.396844 [CLM  ] New Configuration:
> Apr 25 13:15:08.396925 [CLM  ] 	10.2.1.7
> Apr 25 13:15:08.397138 [CLM  ] Members Left:
> Apr 25 13:15:08.397205 [CLM  ] Members Joined:
> Apr 25 13:15:08.397317 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:08.397387 [CLM  ] New Configuration:
> Apr 25 13:15:08.397460 [CLM  ] 	10.2.1.6
> Apr 25 13:15:08.397678 [CLM  ] 	10.2.1.7
> Apr 25 13:15:08.397746 [CLM  ] Members Left:
> Apr 25 13:15:08.397811 [CLM  ] Members Joined:
> Apr 25 13:15:08.397883 [CLM  ] 	10.2.1.6
> Apr 25 13:15:08.397973 [SYNC ] This node is within the non-primary  
> component and will NOT provide any services.
> Apr 25 13:15:08.448519 [TOTEM] entering OPERATIONAL state.
> Apr 25 13:15:09.766290 [YKD  ] This processor is within the primary  
> component.
> Apr 25 13:15:09.766747 [SYNC ] This node is within the primary  
> component and will provide service.
> Apr 25 13:15:09.767387 [YKD  ] This processor is within the primary  
> component.
> Apr 25 13:15:09.767632 [SYNC ] This node is within the primary  
> component and will provide service.
> Apr 25 13:15:09.780704 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:09.780914 [SYNC ] Synchronization actions starting for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:09.781237 [SYNC ] Synchronization actions done for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:09.785445 [CLM  ] got nodejoin message 10.2.1.7
> Apr 25 13:15:09.789360 [CLM  ] got nodejoin message 10.2.1.6
> Apr 25 13:15:09.794258 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:09.794597 [SYNC ] Synchronization actions starting for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:09.795775 [SYNC ] Synchronization actions done for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:09.803874 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:09.804301 [SYNC ] Synchronization actions starting for  
> (openais event service B.01.01)
> Apr 25 13:15:10.804287 [TOTEM] The token was lost in state 1 from  
> timer 83ce000
> Apr 25 13:15:10.805974 [TOTEM] Receive multicast socket recv buffer  
> size (144000 bytes).
> Apr 25 13:15:10.806289 [TOTEM] Transmit multicast socket send buffer  
> size (144000 bytes).
> Apr 25 13:15:10.807411 [TOTEM] entering GATHER state.
> Apr 25 13:15:11.012684 [TOTEM] Saving state aru 73 high seq received 73
> Apr 25 13:15:11.016427 [TOTEM] Storing new sequence id for ring 63752
> Apr 25 13:15:11.016672 [TOTEM] entering COMMIT state.
> Apr 25 13:15:11.020004 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:11.021161 [TOTEM] position [0] member 10.2.1.6:
> Apr 25 13:15:11.021443 [TOTEM] previous ring seq 63748 rep 10.2.1.6
> Apr 25 13:15:11.021523 [TOTEM] aru 73 high delivered 73 received flag 0
> Apr 25 13:15:11.021609 [TOTEM] position [1] member 10.2.1.7:
> Apr 25 13:15:11.021686 [TOTEM] previous ring seq 63748 rep 10.2.1.6
> Apr 25 13:15:11.021760 [TOTEM] aru 73 high delivered 73 received flag 0
> Apr 25 13:15:11.021969 [TOTEM] copying all old ring messages from 74-73.
> Apr 25 13:15:11.022048 [TOTEM] Originated 0 messages in RECOVERY.
> Apr 25 13:15:11.022120 [TOTEM] Originated for recovery:
> Apr 25 13:15:11.022188 [TOTEM] Not Originated for recovery:
> Apr 25 13:15:12.022718 [TOTEM] The token was lost in state 4 from  
> timer 83ce000
> Apr 25 13:15:12.022950 [TOTEM] Restoring instance->my_aru 73 my high  
> seq received 73
> Apr 25 13:15:12.023855 [TOTEM] entering GATHER state.
> Apr 25 13:15:12.225043 [TOTEM] entering GATHER state.
> Apr 25 13:15:12.225916 [TOTEM] Creating commit token because I am the  
> rep.
> Apr 25 13:15:12.226887 [TOTEM] Storing new sequence id for ring 63756
> Apr 25 13:15:12.227084 [TOTEM] entering COMMIT state.
> Apr 25 13:15:12.228023 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:12.229495 [TOTEM] position [0] member 10.2.1.7:
> Apr 25 13:15:12.229831 [TOTEM] previous ring seq 63748 rep 10.2.1.6
> Apr 25 13:15:12.229910 [TOTEM] aru 73 high delivered 73 received flag 0
> Apr 25 13:15:12.230040 [TOTEM] copying all old ring messages from 74-73.
> Apr 25 13:15:12.230266 [TOTEM] Originated 0 messages in RECOVERY.
> Apr 25 13:15:12.230339 [TOTEM] Originated for recovery:
> Apr 25 13:15:12.230406 [TOTEM] Not Originated for recovery:
> Apr 25 13:15:12.230996 [TOTEM] Sending initial ORF token
> Apr 25 13:15:12.237299 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:12.237481 [CLM  ] New Configuration:
> Apr 25 13:15:12.237563 [CLM  ] 	10.2.1.7
> Apr 25 13:15:12.237792 [CLM  ] Members Left:
> Apr 25 13:15:12.237866 [CLM  ] 	10.2.1.6
> Apr 25 13:15:12.237933 [CLM  ] Members Joined:
> Apr 25 13:15:12.238044 [CKPT ] clean_checkpoint_list: List is empty
> Apr 25 13:15:12.238294 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:12.238367 [CLM  ] New Configuration:
> Apr 25 13:15:12.238441 [CLM  ] 	10.2.1.7
> Apr 25 13:15:12.238508 [CLM  ] Members Left:
> Apr 25 13:15:12.238575 [CLM  ] Members Joined:
> Apr 25 13:15:12.238971 [SYNC ] This node is within the non-primary  
> component and will NOT provide any services.
> Apr 25 13:15:12.290823 [TOTEM] entering OPERATIONAL state.
> Apr 25 13:15:12.301039 [YKD  ] This processor is within the primary  
> component.
> Apr 25 13:15:12.301487 [SYNC ] This node is within the primary  
> component and will provide service.
> Apr 25 13:15:12.303028 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:12.303346 [SYNC ] Synchronization actions starting for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:12.303501 [SYNC ] Synchronization actions done for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:12.304808 [CLM  ] got nodejoin message 10.2.1.7
> Apr 25 13:15:12.305985 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:12.306116 [SYNC ] Synchronization actions starting for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:12.306824 [SYNC ] Synchronization actions done for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:12.309345 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:12.309499 [SYNC ] Synchronization actions starting for  
> (openais event service B.01.01)
> Apr 25 13:15:12.314866 [TOTEM] entering GATHER state.
> Apr 25 13:15:12.317401 [TOTEM] Saving state aru 2c high seq received 2c
> Apr 25 13:15:12.318473 [TOTEM] Storing new sequence id for ring 63760
> Apr 25 13:15:12.318853 [TOTEM] entering COMMIT state.
> Apr 25 13:15:12.323519 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:12.324943 [TOTEM] position [0] member 10.2.1.6:
> Apr 25 13:15:12.325107 [TOTEM] previous ring seq 63748 rep 10.2.1.6
> Apr 25 13:15:12.325329 [TOTEM] aru 73 high delivered 73 received flag 0
> Apr 25 13:15:12.325412 [TOTEM] position [1] member 10.2.1.7:
> Apr 25 13:15:12.325487 [TOTEM] previous ring seq 63756 rep 10.2.1.7
> Apr 25 13:15:12.325557 [TOTEM] aru 2c high delivered 2b received flag 0
> Apr 25 13:15:12.325636 [TOTEM] copying all old ring messages from 2d-2c.
> Apr 25 13:15:12.325867 [TOTEM] Originated 0 messages in RECOVERY.
> Apr 25 13:15:12.325937 [TOTEM] Originated for recovery:
> Apr 25 13:15:12.326002 [TOTEM] Not Originated for recovery:
> Apr 25 13:15:13.327013 [TOTEM] The token was lost in state 4 from  
> timer 83ce000
> Apr 25 13:15:13.327242 [TOTEM] Restoring instance->my_aru 2c my high  
> seq received 2c
> Apr 25 13:15:13.328109 [TOTEM] entering GATHER state.
> Apr 25 13:15:13.529877 [TOTEM] entering GATHER state.
> Apr 25 13:15:13.530717 [TOTEM] Creating commit token because I am the  
> rep.
> Apr 25 13:15:13.531687 [TOTEM] Storing new sequence id for ring 63764
> Apr 25 13:15:13.531887 [TOTEM] entering COMMIT state.
> Apr 25 13:15:13.532459 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:13.533777 [TOTEM] position [0] member 10.2.1.7:
> Apr 25 13:15:13.533927 [TOTEM] previous ring seq 63756 rep 10.2.1.7
> Apr 25 13:15:13.534150 [TOTEM] aru 2c high delivered 2b received flag 0
> Apr 25 13:15:13.534232 [TOTEM] copying all old ring messages from 2d-2c.
> Apr 25 13:15:13.534309 [TOTEM] Originated 0 messages in RECOVERY.
> Apr 25 13:15:13.534379 [TOTEM] Originated for recovery:
> Apr 25 13:15:13.534444 [TOTEM] Not Originated for recovery:
> Apr 25 13:15:13.535177 [TOTEM] Sending initial ORF token
> Apr 25 13:15:13.543807 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:13.544203 [CLM  ] New Configuration:
> Apr 25 13:15:13.544284 [CLM  ] 	10.2.1.7
> Apr 25 13:15:13.544352 [CLM  ] Members Left:
> Apr 25 13:15:13.544415 [CLM  ] Members Joined:
> Apr 25 13:15:13.544645 [CLM  ] CLM CONFIGURATION CHANGE
> Apr 25 13:15:13.544718 [CLM  ] New Configuration:
> Apr 25 13:15:13.544790 [CLM  ] 	10.2.1.7
> Apr 25 13:15:13.544856 [CLM  ] Members Left:
> Apr 25 13:15:13.544921 [CLM  ] Members Joined:
> Apr 25 13:15:13.545122 [SYNC ] This node is within the non-primary  
> component and will NOT provide any services.
> Apr 25 13:15:13.594240 [TOTEM] entering OPERATIONAL state.
> Apr 25 13:15:13.604969 [YKD  ] This processor is within the primary  
> component.
> Apr 25 13:15:13.605348 [SYNC ] This node is within the primary  
> component and will provide service.
> Apr 25 13:15:13.606757 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:13.606908 [SYNC ] Synchronization actions starting for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:13.607207 [SYNC ] Synchronization actions done for  
> (openais cluster membership service B.01.01)
> Apr 25 13:15:13.608359 [CLM  ] got nodejoin message 10.2.1.7
> Apr 25 13:15:13.609707 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:13.609839 [SYNC ] Synchronization actions starting for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:13.609962 [SYNC ] Synchronization actions done for  
> (openais checkpoint service B.01.01)
> Apr 25 13:15:13.612200 [SYNC ] Synchronization barrier completed
> Apr 25 13:15:13.612341 [SYNC ] Synchronization actions starting for  
> (openais event service B.01.01)
> Apr 25 13:15:13.616743 [SYNC ] Synchronization actions done for  
> (openais event service B.01.01)
> Apr 25 13:15:13.645939 [TOTEM] entering GATHER state.
> Apr 25 13:15:13.648425 [TOTEM] Saving state aru 2b high seq received 2b
> Apr 25 13:15:13.649434 [TOTEM] Storing new sequence id for ring 63768
> Apr 25 13:15:13.649811 [TOTEM] entering COMMIT state.
> Apr 25 13:15:13.654423 [TOTEM] entering RECOVERY state.
> Apr 25 13:15:13.655888 [TOTEM] position [0] member 10.2.1.6:
> Apr 25 13:15:13.656440 [TOTEM] previous ring seq 63748 rep 10.2.1.6
> Apr 25 13:15:13.656544 [TOTEM] aru 73 high delivered 73 received flag 0
> Apr 25 13:15:13.658976 [TOTEM] position [1] member 10.2.1.7:
> Apr 25 13:15:13.659273 [TOTEM] previous ring seq 63764 rep 10.2.1.7
> Apr 25 13:15:13.659355 [TOTEM] aru 2b high delivered 2b received flag 0
> Apr 25 13:15:13.659438 [TOTEM] copying all old ring messages from 2c-2b.
> Apr 25 13:15:13.659514 [TOTEM] Originated 0 messages in RECOVERY.
> Apr 25 13:15:13.659714 [TOTEM] Originated for recovery:
> Apr 25 13:15:13.659785 [TOTEM] Not Originated for recovery:
> Apr 25 13:15:14.659438 [TOTEM] The token was lost in state 4 from  
> timer 83ce000
> Apr 25 13:15:14.659700 [TOTEM] Restoring instance->my_aru 2b my high  
> seq received 2b
> Apr 25 13:15:14.660615 [TOTEM] entering GATHER state.
> ...
> 
> _______________________________________________
> Openais mailing list
> Openais at lists.osdl.org
> https://lists.osdl.org/mailman/listinfo/openais




More information about the Openais mailing list