[Openais] clvmd + openais "Cannot allocate memory"

[Christine Caulfield]

On 28/04/10 05:37, Quentin Smith wrote:
> [please preserve the Cc in replies]
>
> Hi-
>
> I'm trying to debug a problem that has surfaced recently in our clvmd
> 2.02.56 + openais 1.1.1 + corosync 1.1.2 cluster. We use clvmd to manage
> volumes on a shared iSCSI storage array. We have four nodes in our
> cluster, and after running for a while we start getting errors when trying
> to perform LVM operations such as:
>
> root at babylon-four:~# lvs
>     cluster request failed: Cannot allocate memory
>     Can't get lock for xenvg
>     Skipping volume group xenvg
>     cluster request failed: Cannot allocate memory
>     Can't get lock for babylon-four
>
> and
>
> root at citadel-station:~# lvcreate -L 1G -n quentin-test-2 xenvg
>     Error locking on node 1280050a: Cannot allocate memory
>     Error locking on node 1380050a: Cannot allocate memory
>     Error locking on node 1180050a: Cannot allocate memory
>     Error locking on node 1080050a: Cannot allocate memory
>     Aborting. Failed to activate new LV to wipe the start of it.
>
> We had been able to clear these errors up by restarting corosync and
> clvmd, but now that we've added another two nodes to our cluster, we're
> getting these errors very frequently. Is there a buffer somewhere that we
> need to increase the size of? I've attached our current corosync.conf to
> the end of this message.
>
> (Needless to say, the servers all have plenty of RAM free...)
>

There was a memory leak in clvmd up to 2.0.56 that might be having an 
impact here if there are a lot of clvmd operation being done. eg. 
monitoring software that polls active LVs frequently.

That's all I can think of off-hand that might cause that

Chrissie