Re: dynamic shared memory

On Sun, Sep 01, 2013 at 05:08:38PM +0200, Andres Freund wrote:
> On 2013-09-01 09:24:00 -0400, Noah Misch wrote:
> > The difficulty depends on whether processes other than the segment's creator
> > will attach anytime or only as they start. Attachment at startup is enough
> > for parallel query, but it's not enough for something like lock table
> > expansion. I'll focus on the attach-anytime case since it's more general.
>
> Even on startup it might get more complicated than one immediately
> imagines on EXEC_BACKEND type platforms because their memory layout
> doesn't need to be the same. The more shared memory you need, the harder
> that will be. Afair

Non-Windows EXEC_BACKEND is already facing a dead end that way.

> > On a system supporting MAP_FIXED, implement this by having the postmaster
> > reserve address space under a PROT_NONE mapping, then carving out from that
> > mapping for each fixed-address dynamic segment. The size of the reservation
> > would be controlled by a GUC; one might set it to several times anticipated
> > peak usage. (The overhead of doing that depends on the kernel.) Windows
> > permits the same technique with its own primitives.
>
> Note that allocating a large mapping, even without using it, has
> noticeable cost, at least under linux. The kernel has to create & copy
> data to track each pages state (without copying the memory content's
> itself due to COW) for every fork afterwards. If you don't believe me,
> check the whole discussion about go's (the language) memory
> management...

I believe you, but I'd appreciate a link to the discussion you have in mind.

> If that's the solution we go for why don't we just always include heaps
> more shared memory and use that (remapping part of it as PROT_NONE)
> instead of building the infrastructure in this patch?

There would be no freeing of the memory; a usage high water mark would stand
for the life of the postmaster.

> > I don't foresee fundamental differences on 32-bit. All the allocation
> > maximums scale down, but that's the usual story for 32-bit.
>
> If you actually want to allocate memory after starting up, without
> carving a section out for that from the beginning, the memory
> fragmentation will make it very hard to find memory addresses of the
> same across processes.

True. I wouldn't feel bad if total dynamic shared memory usage above, say,
256 MiB were unreliable on 32-bit. If you're still running 32-bit in 2015,
you probably have a low-memory platform.

I think the take-away is that we have a lot of knobs available, not a bright
line between possible and impossible. Robert opted to omit provision for
reliable fixed addresses, and the upsides of that decision are the absence of
a DBA-unfriendly space-reservation GUC, trivial overhead when the APIs are not
used, and a clearer portability outlook.

--
Noah Misch
EnterpriseDB http://www.enterprisedb.com