Re: dynamically allocating chunks from shared memory

Markus Wanner wrote:
> Hi,
>
> On 08/09/2010 06:31 PM, Bruce Momjian wrote:
> > Let me be more concrete. Suppose you are using threads, and you want to
> > increase your shared memory from 20MB to 30MB. How do you do that?
>
> There's absolutely no need to pre-allocate 20 MB in advance in a
> threaded environment. You just allocate memory in small chunks. For a
> threaded-model, that memory is shared by default, so the total amount of
> shared memory can grow and shrink very easily. (And even makes usused
> memory available to other processes, not just other threads).
>
> > If
> > you want it contiguous, you have to use realloc, which might move the
> > pointer. If you allocate another 10MB chunk, you then have shared
> > memory fragments, which is the same as adding another shared memory
> > segment.
>
> Okay, I think I now understand the requirement of continuity you
> mentioned earlier already. I agree that with the current approach, we
> cannot simply use such a dynamic allocator to solve all of our problems.
>
> Every subsystem would need to be converted to something that allocates
> shared memory in smaller chunks for such a dynamic allocator to be of
> any use. Robert already pointed out that this may be troublesome for
> shared_buffers, which is by far the largest consumer of shared memory. I
> didn't look into this, yet. And I'd like to hear more about the
> feasibility of that approach for other subsystems.
>
> Another issue to be discussed would be the limits of sharing free memory
> between subsystems. Maybe we even reach the conclusion that we
> absolutely *want* fixed maximum sizes for every single subsystem so as
> to be able to guarantee a certain amount of multi-xact or SLRU entries
> at any point in time (otherwise one memory hungry subsystem could
> possibly eat it all up with another subsystem getting the OOM error when
> trying to allocate for its very first entry).

Yep, you would have to use chunks in threads/malloc, and you have to do
the same thing with shared memory.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +