Re: initdb fails to allocate shared memory

I am using pgsql9.0b4 (but pgsql8.4 exhibits the same behavior) on MacOS 10.6.4 and initdb fails:

/usr/local/pgsql90beta/bin/initdb -D /Volumes/Data/pgsql90b/ -E UTF8
The files belonging to this database system will be owned by user "agentm".
This user must also own the server process.

The database cluster will be initialized with locale en_US.UTF-8.
The default text search configuration will be set to "english".

creating directory /Volumes/Data/pgsql90b ... ok
creating subdirectories ... ok
selecting default max_connections ... 10
selecting default shared_buffers ... 400kB
creating configuration files ... ok
creating template1 database in /Volumes/Data/pgsql90b/base/1 ... FATAL: could not create shared memory segment: Cannot allocate memory
DETAIL: Failed system call was shmget(key=1, size=1703936, 03600).
HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory or swap space. To reduce the request size (currently 1703936 bytes), reduce PostgreSQL's shared_buffers parameter (currently 50) and/or its max_connections parameter (currently 14).
The PostgreSQL documentation contains more information about shared memory configuration.
child process exited with exit code 1
initdb: removing data directory "/Volumes/Data/pgsql90b"

I would like to create the database space and then reduce the shared memory requirements in postgresql.conf, but this situation seems to create a chicken-and-egg problem. How can I reduce shared_buffers or max_connections prior to running initdb?

Cheers,
M

On Wed, 2010-08-25 at 11:15 -0400, A.M. wrote:
> I am using pgsql9.0b4 (but pgsql8.4 exhibits the same behavior) on MacOS 10.6.4 and initdb fails:
> initdb: removing data directory "/Volumes/Data/pgsql90b"
>
> I would like to create the database space and then reduce the shared memory requirements in postgresql.conf, but this situation seems to create a chicken-and-egg problem. How can I reduce shared_buffers or max_connections prior to running initdb?

If you don't have enough shared memory to initdb, you don't have enough
to run postgresql. You need to increase your shared memory for MacOS
per:

http://www.postgresql.org/docs/8.4/static/kernel-resources.html

And then initdb.

Sincerely,

Joshua D. Drake

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579
Consulting, Training, Support, Custom Development, Engineering
http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt

On Aug 25, 2010, at 11:31 AM, Joshua D. Drake wrote:

> On Wed, 2010-08-25 at 11:15 -0400, A.M. wrote:
>> I am using pgsql9.0b4 (but pgsql8.4 exhibits the same behavior) on MacOS 10.6.4 and initdb fails:
>> initdb: removing data directory "/Volumes/Data/pgsql90b"
>>
>> I would like to create the database space and then reduce the shared memory requirements in postgresql.conf, but this situation seems to create a chicken-and-egg problem. How can I reduce shared_buffers or max_connections prior to running initdb?
>
> If you don't have enough shared memory to initdb, you don't have enough
> to run postgresql. You need to increase your shared memory for MacOS
> per:
>
> http://www.postgresql.org/docs/8.4/static/kernel-resources.html
>
> And then initdb.

Then it seems that the error reporting could be improved to not mention "shared_buffers" and "max_connections" neither of which I can touch during initdb.

"creating template1 database in /Volumes/Data/pgsql90b/base/1 ... FATAL: could not create shared memory segment: Cannot allocate memory
DETAIL: Failed system call was shmget(key=1, size=1703936, 03600).
HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory or swap space. To reduce the request size (currently 1703936 bytes), reduce PostgreSQL's shared_buffers parameter (currently 50) and/or its max_connections parameter (currently 14).
The PostgreSQL documentation contains more information about shared memory configuration."

Cheers,
M

"A.M." <agentm(at)themactionfaction(dot)com> writes:
> Then it seems that the error reporting could be improved to not mention "shared_buffers" and "max_connections" neither of which I can touch during initdb.

The error has to be phrased to cover the case where you hit it after
initdb. It would be quite unhelpful to *not* mention those settings.

However, it's odd that you got this variant of the HINT and not the one
that suggests increasing SHMMAX. Looking at the code, that means that
shmget returned ENOMEM, not EINVAL, which surprises me. What did you
have your kernel settings at? Could we see the output of
sysctl -a | grep sysv

regards, tom lane

I wrote:
> Could we see the output of
> sysctl -a | grep sysv

"ipcs -a" might be informative, too.

regards, tom lane

"A.M." <agentm(at)themactionfaction(dot)com> writes:
> On Aug 25, 2010, at 12:28 PM, Tom Lane wrote:
>> However, it's odd that you got this variant of the HINT and not the one
>> that suggests increasing SHMMAX. Looking at the code, that means that
>> shmget returned ENOMEM, not EINVAL, which surprises me.

> I was already running an instance of pgsql 8.4, so that sucked up all
> the shared mem space and there was none left for 9.0b4.

Ah-hah. I now realize that we've been interpreting ENOMEM too narrowly.
The Single Unix Spec defines ENOMEM from shmget thusly:

[ENOMEM]
A shared memory identifier and associated shared memory segment
are to be created but the amount of available physical memory is
not sufficient to fill the request.

from which you'd assume that the only way to fix it is (a) buy more RAM
or (b) decrease the request size. Thus we only suggest (b). However,
it appears from your report that OS X is also using ENOMEM when SHMALL
is exceeded, which is not all that surprising because actually none of
the spec-defined error codes cover SHMALL exhaustion. The Linux manpage
for shmget says that ENOSPC is used when SHMALL is exhausted, which is
equally reasonable but it's *not* what we read in SUS:

[ENOSPC]
A shared memory identifier is to be created but the
system-imposed limit on the maximum number of allowed shared
memory identifiers system-wide would be exceeded.

So apparently different implementations are not very consistent about
which code they return for SHMALL exhaustion, and we should change the
wording for the ENOMEM hint to mention SHMALL as a possible cause.

regards, tom lane

I wrote:
> it appears from your report that OS X is also using ENOMEM when SHMALL
> is exceeded, which is not all that surprising because actually none of
> the spec-defined error codes cover SHMALL exhaustion.

A look into
http://www.opensource.apple.com/source/xnu/xnu-1504.7.4/bsd/kern/sysv_shm.c
confirms this:

if ((user_ssize_t)(shm_committed + btoc(total_size)) > shminfo.shmall)
return ENOMEM;

So this behavior is probably common among BSDen.

regards, tom lane