Re: POSIX shared memory redux

On Apr 11, 2011, at 7:13 PM, Tom Lane wrote:

> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Mon, Apr 11, 2011 at 3:11 PM, A.M. <agentm(at)themactionfaction(dot)com> wrote:
>>> What do you mean by "leakier"? The goal here is to extinguish SysV shared memory for portability and convenience benefits. The mini-SysV proposal was implemented and shot down by Tom Lane.
>
>> I mean I'm not convinced that fcntl() locking will be as reliable.
>
> I'm not either. Particularly not on NFS. (Although on NFS you have
> other issues to worry about too, like postmasters on different machines
> being able to reach the same data directory. I wonder if we should do
> both SysV and fcntl locking ...)

Is there an example of a recent system where fcntl is broken (ignoring NFS)? I believe my patch addresses all potential race conditions and uses the APIs properly to guarantee single-postmaster data directory usage and I tested on Darwin and a two-year-old Linux kernel. In the end, fcntl locking relies on the same kernel which provides the SysV user count, so I'm not sure what makes it less "reliable", but I have heard that twice now, so I am open to hearing about your experiences.

>> I know Tom shot that down before, but I still think it's probably the
>> best way forward.
>
> Did I? I think I pointed out that there's zero gain in portability as
> long as we still depend on SysV shmem to work. However, if you're doing
> it for other reasons than portability, it might make sense anyway. The
> question is whether there are adequate other reasons.

I provided an example of postmaster-failover relying on F_SETLKW in the email with the patch. Also, as you point out above, fcntl locking at least has a chance of working over NFS.

>
>> The advantage I see is that we would be able to
>> more easily allocate larger chunks of shared memory with changing
>> kernel parameters,
>
> Yes, getting out from under the SHMMAX bugaboo would be awfully nice.

Yes, please! That is my primary motivation for this patch.

>
>> and perhaps even to dynamically resize shared memory chunks.
>
> This I don't really believe will ever work reliably, especially not in
> 32-bit machines. Whatever your kernel API is, you still have the
> problem of finding address space contiguous to what you were already
> using.

Even if expanding shmem involves copying large regions of memory, it could be at least useful to adjust buffer sizes live without a restart.

Cheers,
M