Re: [HACKERS] ipc-daemon

I'm getting tired of the "initdb hangs" class of complaints. Why doesn't
the relevant function fail if the ipc-daemon isn't running? Can anything
be done in that area?

--
Peter Eisentraut peter_e(at)gmx(dot)net

Peter,

On Wed, Oct 30, 2002 at 08:35:41PM +0100, Peter Eisentraut wrote:
> I'm getting tired of the "initdb hangs" class of complaints. Why
> doesn't the relevant function fail if the ipc-daemon isn't running?
> Can anything be done in that area?

It is usually more productive to debug and/or provide a patch than to
complain. Remember, this is open source and not proprietary software.
If you have the itch, then scratch it.

Nevertheless, I have debugged the root cause of this problem:

1. If ipc-daemon is not running, then cygipc's shmget() will return
EACCES.
2. This causes PostgreSQL's InternalIpcMemoryCreate() to return
NULL.
3. This causes PostgreSQL's PGSharedMemoryCreate() to spin looking
for accessible shared memory which it will never find.

With the attached cygipc patch (against cygipc 1.11-1), postmaster fails
when ipc-daemon is not running as follows:

$ postmaster -D /usr/share/postgresql/data
IpcMemoryCreate: shmget(key=5432001, size=1441792, 03600) failed: No space left on device

This error does *not* mean that you have run out of disk space.

It occurs either if all available shared memory IDs have been taken,
in which case you need to raise the SHMMNI parameter in your kernel,
or because the system's overall limit for shared memory has been
reached. If you cannot increase the shared memory limit,
reduce PostgreSQL's shared memory request (currently 1441792 bytes),
by reducing its shared_buffers parameter (currently 64) and/or
its max_connections parameter (currently 32).

The PostgreSQL Administrator's Guide contains more information about
shared memory configuration.

Is the above an acceptable solution?

After reading the shmget() man page, I choose to return ENOSPC. Is
there a better choice? Such as ENOMEM?

Note that I'm not very knowledgeable in the area of shared memory and
can use some help determining the best error to return when ipc-daemon
is not running.

Once we reach consensus on the above, I will submit a patch to the
cygipc maintainer. Unfortunately, he is currently defending his Ph.D
thesis and has indicated that he will not deal with Cygwin issues until
he is finished.

Jason

Peter,

On Thu, Oct 31, 2002 at 03:18:31PM -0500, Jason Tishler wrote:
> On Wed, Oct 30, 2002 at 08:35:41PM +0100, Peter Eisentraut wrote:
> > I'm getting tired of the "initdb hangs" class of complaints. Why
> > doesn't the relevant function fail if the ipc-daemon isn't running?
> > Can anything be done in that area?

The attached cygipc patch handles the above "initdb hang" and is very
similar to my previous one which handles the "postmaster hang." With
this patch applied, we now get the following:

$ initdb -D /usr/share/postgresql/data
The files belonging to this database system will be owned by user "jt".
This user must also own the server process.

The database cluster will be initialized with locale C.

creating directory /usr/share/postgresql/data... ok
[snip]
creating template1 database in /usr/share/postgresql/data/base/1... IpcSemaphoreCreate: semget(key=1, num=17, 03600) failed: No space left on device

This error does *not* mean that you have run out of disk space.

It occurs when either the system limit for the maximum number of
semaphore sets (SEMMNI), or the system wide maximum number of
semaphores (SEMMNS), would be exceeded. You need to raise the
respective kernel parameter. Alternatively, reduce PostgreSQL's
consumption of semaphores by reducing its max_connections parameter
(currently 1).

The PostgreSQL Administrator's Guide contains more information about
configuring your system for PostgreSQL.

initdb failed.
Removing /usr/share/postgresql/data.

The same questions and caveats apply as in my previous post.

There is one other case that I haven't handled yet. cygipc's
msg_connect() can fail in a similar way. Once the disposition of the
initdb and postmaster hang patches is known, I will tackle this one (if
necessary).

Nevertheless, with these two patches, I believe that I have addressed
your concerns. Do you agree?

Jason

Jason Tishler writes:

> 1. If ipc-daemon is not running, then cygipc's shmget() will return
> EACCES.
> 2. This causes PostgreSQL's InternalIpcMemoryCreate() to return
> NULL.
> 3. This causes PostgreSQL's PGSharedMemoryCreate() to spin looking
> for accessible shared memory which it will never find.

To me, this is a bug in PostgreSQL. A comment in
InternalIpcMemoryCreate() says

* Fail quietly if error indicates a collision with existing
* segment. One would expect EEXIST, given that we said IPC_EXCL,
* but perhaps we could get a permission violation instead?

I tend to think that the answer to that question is No.

> After reading the shmget() man page, I choose to return ENOSPC. Is
> there a better choice? Such as ENOMEM?

My first thought was ENOSYS (system call not implemented -- what BSD
kernels tend to return if you didn't compile them with SysV IPC support),
but that isn't a clearly superior choice either. Fixing PostgreSQL is
probably better and quicker to yield a return.

--
Peter Eisentraut peter_e(at)gmx(dot)net

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Jason Tishler writes:
>> 1. If ipc-daemon is not running, then cygipc's shmget() will return
>> EACCES.
>> 2. This causes PostgreSQL's InternalIpcMemoryCreate() to return
>> NULL.
>> 3. This causes PostgreSQL's PGSharedMemoryCreate() to spin looking
>> for accessible shared memory which it will never find.

> To me, this is a bug in PostgreSQL.

I disagree: just because cygipc returns error codes chosen at random
doesn't mean that we should neglect the clear meaning of an error code.
If a normal Unix system were to return EACCES here, the clear
implication would be that there is an existing segment that we do not
have permission to access. I don't see how "cygipc isn't running"
can reasonably be translated into "permission denied".

>> After reading the shmget() man page, I choose to return ENOSPC. Is
>> there a better choice? Such as ENOMEM?

> My first thought was ENOSYS (system call not implemented -- what BSD
> kernels tend to return if you didn't compile them with SysV IPC support),
> but that isn't a clearly superior choice either.

If you can detect that cygipc is not running, then ENOSYS seems the
best choice for reporting that. (ENOSPC would be misleading too.)

If it's impractical to fix cygipc then I'd grudgingly go along with

if (errno == EEXIST
#ifndef __CYWGIN__ /* cygipc is broken */
|| errno == EACCES
#endif
#ifdef EIDRM
|| errno == EIDRM
#endif
)
return NULL;

regards, tom lane

Tom,
Peter,

On Mon, Nov 04, 2002 at 02:43:01PM -0500, Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > To me, this is a bug in PostgreSQL.
>
> I disagree: just because cygipc returns error codes chosen at random
> doesn't mean that we should neglect the clear meaning of an error
> code. If a normal Unix system were to return EACCES here, the clear
> implication would be that there is an existing segment that we do not
> have permission to access. I don't see how "cygipc isn't running" can
> reasonably be translated into "permission denied".
>
> [snip]
>
> > My first thought was ENOSYS (system call not implemented -- what BSD
> > kernels tend to return if you didn't compile them with SysV IPC
> > support), but that isn't a clearly superior choice either.
>
> If you can detect that cygipc is not running, then ENOSYS seems the
> best choice for reporting that. (ENOSPC would be misleading too.)
>
> If it's impractical to fix cygipc then I'd grudgingly go along with
>
> if (errno == EEXIST
> #ifndef __CYWGIN__ /* cygipc is broken */
> || errno == EACCES
> #endif
> #ifdef EIDRM
> || errno == EIDRM
> #endif
> )
> return NULL;

Thanks for your feedback. I will take this to the Cygwin list and see
what happens. Unfortunately, the cygipc maintainer is "AWOL" right now
because of Ph.D. thesis commitments. Hence, even if I can get the
Cygwin community to agree to this change, there may not be an official
cygipc release for a while.

Thanks,
Jason

--
GPG key available on key servers or http://www.tishler.net/jason/gpg.txt

Tom Lane writes:

> I disagree: just because cygipc returns error codes chosen at random
> doesn't mean that we should neglect the clear meaning of an error code.
> If a normal Unix system were to return EACCES here, the clear
> implication would be that there is an existing segment that we do not
> have permission to access.

OK, but shouldn't the looping for a new segment terminate sometime? As it
stands it will wrap around and run forever. If we catch it before it
wraps and generate an error to the effect that no segments appear to be
free, it might serve us.

--
Peter Eisentraut peter_e(at)gmx(dot)net

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Tom Lane writes:
>> I disagree: just because cygipc returns error codes chosen at random
>> doesn't mean that we should neglect the clear meaning of an error code.
>> If a normal Unix system were to return EACCES here, the clear
>> implication would be that there is an existing segment that we do not
>> have permission to access.

> OK, but shouldn't the looping for a new segment terminate sometime? As it
> stands it will wrap around and run forever.

Hm. It would be reasonable to put a limit on the number of attempts,
probably. (There's a similar limit on attempts to create a lockfile.)

regards, tom lane

Tom,
Peter,

On Mon, Nov 04, 2002 at 03:05:25PM -0500, Jason Tishler wrote:
> On Mon, Nov 04, 2002 at 02:43:01PM -0500, Tom Lane wrote:
> > If you can detect that cygipc is not running, then ENOSYS seems the
> > best choice for reporting that. (ENOSPC would be misleading too.)
>
> Thanks for your feedback. I will take this to the Cygwin list and see
> what happens.

I happy to report that the above has been accomplished:

http://cygwin.com/ml/cygwin-announce/2002-11/msg00025.html

Hopefully, the Cygwin initdb and postmaster hang posts will be a thing
of the past. Unfortunately, they probably will be replaced by questions
like:

Why does initdb fail with "IpcMemoryCreate: shmget(...) failed:
Function not implemented?"

Jason

--
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6

At least that is an FAQ item.

---------------------------------------------------------------------------

Jason Tishler wrote:
> Tom,
> Peter,
>
> On Mon, Nov 04, 2002 at 03:05:25PM -0500, Jason Tishler wrote:
> > On Mon, Nov 04, 2002 at 02:43:01PM -0500, Tom Lane wrote:
> > > If you can detect that cygipc is not running, then ENOSYS seems the
> > > best choice for reporting that. (ENOSPC would be misleading too.)
> >
> > Thanks for your feedback. I will take this to the Cygwin list and see
> > what happens.
>
> I happy to report that the above has been accomplished:
>
> http://cygwin.com/ml/cygwin-announce/2002-11/msg00025.html
>
> Hopefully, the Cygwin initdb and postmaster hang posts will be a thing
> of the past. Unfortunately, they probably will be replaced by questions
> like:
>
> Why does initdb fail with "IpcMemoryCreate: shmget(...) failed:
> Function not implemented?"
>
> Jason
>
> --
> PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
> Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Just an explanation in the FAQ that the ipc-daemon is not running won't suffice. Because in my case I had ipc-daemon (version 1.11) running and it still hung (Jason's patch reported the IpcMemoryCreate error). Only when I downgraded to version 1.09 (office) and upgraded to 1.13 (home) did initdb succeed. So I'd suggest also covering this scenario.
Thanks,
Richard
Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> wrote:
At least that is an FAQ item.

---------------------------------------------------------------------------

Jason Tishler wrote:
> Tom,
> Peter,
>
> On Mon, Nov 04, 2002 at 03:05:25PM -0500, Jason Tishler wrote:
> > On Mon, Nov 04, 2002 at 02:43:01PM -0500, Tom Lane wrote:
> > > If you can detect that cygipc is not running, then ENOSYS seems the
> > > best choice for reporting that. (ENOSPC would be misleading too.)
> >
> > Thanks for your feedback. I will take this to the Cygwin list and see
> > what happens.
>
> I happy to report that the above has been accomplished:
>
> http://cygwin.com/ml/cygwin-announce/2002-11/msg00025.html
>
> Hopefully, the Cygwin initdb and postmaster hang posts will be a thing
> of the past. Unfortunately, they probably will be replaced by questions
> like:
>
> Why does initdb fail with "IpcMemoryCreate: shmget(...) failed:
> Function not implemented?"
>
> Jason
>
> --
> PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
> Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

---------------------------------
Do you Yahoo!?
Yahoo! Web Hosting - Let the expert host your site

We normally don't go into that much detail in the FAQ unless someone is
seeing that problem case a lot.

---------------------------------------------------------------------------

Richard Pais wrote:
>
> Just an explanation in the FAQ that the ipc-daemon is not running won't suffice. Because in my case I had ipc-daemon (version 1.11) running and it still hung (Jason's patch reported the IpcMemoryCreate error). Only when I downgraded to version 1.09 (office) and upgraded to 1.13 (home) did initdb succeed. So I'd suggest also covering this scenario.
> Thanks,
> Richard
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> wrote:
> At least that is an FAQ item.
>
> ---------------------------------------------------------------------------
>
> Jason Tishler wrote:
> > Tom,
> > Peter,
> >
> > On Mon, Nov 04, 2002 at 03:05:25PM -0500, Jason Tishler wrote:
> > > On Mon, Nov 04, 2002 at 02:43:01PM -0500, Tom Lane wrote:
> > > > If you can detect that cygipc is not running, then ENOSYS seems the
> > > > best choice for reporting that. (ENOSPC would be misleading too.)
> > >
> > > Thanks for your feedback. I will take this to the Cygwin list and see
> > > what happens.
> >
> > I happy to report that the above has been accomplished:
> >
> > http://cygwin.com/ml/cygwin-announce/2002-11/msg00025.html
> >
> > Hopefully, the Cygwin initdb and postmaster hang posts will be a thing
> > of the past. Unfortunately, they probably will be replaced by questions
> > like:
> >
> > Why does initdb fail with "IpcMemoryCreate: shmget(...) failed:
> > Function not implemented?"
> >
> > Jason
> >
> > --
> > PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
> > Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 2: you can get off all lists at once with the unregister command
> > (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
> >
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania 19073
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/users-lounge/docs/faq.html
>
>
> ---------------------------------
> Do you Yahoo!?
> Yahoo! Web Hosting - Let the expert host your site

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Richard,

On Sat, Nov 16, 2002 at 02:27:22PM -0800, Richard Pais wrote:
> Just an explanation in the FAQ that the ipc-daemon is not running
> won't suffice. Because in my case I had ipc-daemon (version 1.11)
> running and it still hung (Jason's patch reported the IpcMemoryCreate
> error). Only when I downgraded to version 1.09 (office) and upgraded
> to 1.13 (home) did initdb succeed. So I'd suggest also covering this
> scenario.

If you feel strongly about the above, then submit a patch to
pgsql-patches@ for consideration.

Jason

--
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6

To get into the FAQ, it should be something that happens _frequently_,
hence FAQ. Fact is, cygwin may not even be needed in 7.4 because we are
working on a native port.

---------------------------------------------------------------------------

Jason Tishler wrote:
> Richard,
>
> On Sat, Nov 16, 2002 at 02:27:22PM -0800, Richard Pais wrote:
> > Just an explanation in the FAQ that the ipc-daemon is not running
> > won't suffice. Because in my case I had ipc-daemon (version 1.11)
> > running and it still hung (Jason's patch reported the IpcMemoryCreate
> > error). Only when I downgraded to version 1.09 (office) and upgraded
> > to 1.13 (home) did initdb succeed. So I'd suggest also covering this
> > scenario.
>
> If you feel strongly about the above, then submit a patch to
> pgsql-patches@ for consideration.
>
> Jason
>
> --
> PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
> Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Jason Tishler wrote:
> Bruce,
>
> On Mon, Nov 18, 2002 at 10:42:30AM -0500, Bruce Momjian wrote:
> > To get into the FAQ, it should be something that happens _frequently_,
> > hence FAQ.
>
> Understood. That is why is said "for consideration."
>
> > Fact is, cygwin may not even be needed in 7.4 because we are
> > working on a native port.
>
> A Win32 native port in the 7.4 time frame is great news! However, I
> hope that the Cygwin port will not be deprecated. There are compelling
> reasons to still have a Cygwin PostgreSQL port just like there is with
> Python, Perl, Vim, Emacs, etc. On the other hand, I wouldn't mind being
> put out of the Cygwin PostgreSQL support business... :,)

I see no reason to remove cygwin support. I assume they will both
continue to exist.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce,

On Mon, Nov 18, 2002 at 10:42:30AM -0500, Bruce Momjian wrote:
> To get into the FAQ, it should be something that happens _frequently_,
> hence FAQ.

Understood. That is why is said "for consideration."

> Fact is, cygwin may not even be needed in 7.4 because we are
> working on a native port.

A Win32 native port in the 7.4 time frame is great news! However, I
hope that the Cygwin port will not be deprecated. There are compelling
reasons to still have a Cygwin PostgreSQL port just like there is with
Python, Perl, Vim, Emacs, etc. On the other hand, I wouldn't mind being
put out of the Cygwin PostgreSQL support business... :,)

Jason

--
PGP/GPG Key: http://www.tishler.net/jason/pubkey.asc or key servers
Fingerprint: 7A73 1405 7F2B E669 C19D 8784 1AFD E4CC ECF4 8EF6

Bruce Momjian writes:

> To get into the FAQ, it should be something that happens _frequently_,

Just check the pgsql-cygwin archives. We really need a separate list of
Constantly Asked Questions for this.

--
Peter Eisentraut peter_e(at)gmx(dot)net