Re: POSIX question

Florian Pflug <fgp(at)phlo(dot)org> Monday 20 of June 2011 17:01:40
> On Jun20, 2011, at 16:39 , Radosław Smogura wrote:
> > Florian Pflug <fgp(at)phlo(dot)org> Monday 20 of June 2011 16:16:58
> >
> >> On Jun20, 2011, at 15:27 , Radosław Smogura wrote:
> >>> 1. mmap some large amount of anonymous virtual memory (this will be
> >>> maximum size of shared memory). ...
> >>> Point 1. will no eat memory, as memory allocation is delayed and in
> >>> 64bit platforms you may reserve quite huge chunk of this, and in
> >>> future it may be possible using mmap / munmap to concat chunks /
> >>> defrag it etc.
> >>
> >> I think this breaks with strict overcommit settings (i.e.
> >> vm.overcommit_memory = 2 on linux). To fix that, you'd need a way to
> >> tell the kernel (or glibc) to simply reserve a chunk of virtual address
> >> space for further user. Not sure if there's a API for that...
> >>
> >> best regards,
> >> Florian Pflug
> >
> > This may be achived by many other things, like mmap /dev/null.
>
> Are you sure? Isn't mmap()ing /dev/null a way to *allocate* memory?
>
> Or at least this is what I always thought glibc does when you malloc()
> are large block at once. (This allows it to actually return the memory
> to the kernel once you free() it, which isn't possible if the memory
> was allocated simply by extending the heap).
>
> You can work around this by mmap()ing an actual file, because then
> the kernel knows it can use the file as backing store and thus doesn't
> need to reserve actual physical memory. (In a way, this just adds
> additional swap space). Doesn't seem very clean though...
>
> Even if there's a way to work around a strict overcommit setting, unless
> the workaround is a syscall *explicitly* designed for that purpose, I'd
> be very careful with using it. You might just as well be exploiting a
> bug in the overcommit accounting logic and future kernel versions may
> simply choose to fix the bug...
>
> best regards,
> Florian Pflug

I'm sure at 99%. When I ware "playing" with mmap I preallocated, probably,
about 100GB of memory.

Regards,
Radek