Re: Something fishy happening on frogmouth

On Wed, Oct 30, 2013 at 9:49 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
>> On 2013-10-30 09:26:42 -0400, Tom Lane wrote:
>>> Isn't this complaining about the main shm segment, not a DSM extension?
>
>> Don't think so, that has a ":" in the name.
>
> If it *isn't* about the main memory segment, what the hell are we doing
> creating random addon segments during bootstrap? None of the DSM code
> should even get control at this point, IMO.

Here's a short summary of what I posted back in August: at system
startup time, the postmaster creates one dynamic shared segment,
called the control segment. That segment sticks around for the
lifetime of the server and records the identity of any *other* dynamic
shared memory segments that are subsequently created. If the server
dies a horrible death (e.g. kill -9), the next postmaster will find
the previous control segment (whose ID is written to a file in the
data directory) and remove any leftover shared memory segments from
the previous run; without this, such segments would live until the
next server reboot unless manually removed by the user (which isn't
even practical on all platforms; e.g. there doesn't seem to be any way
to list all exstant POSIX shared memory segments on MacOS X, so a user
wouldn't know which segments to remove).

For my previous posting on this topic, see the following link,
particularly the paragraph which begins "The actual implementation is
split up into two layers" and the following one.

http://www.postgresql.org/message-id/CA+TgmoaDqDUgt=4Zs_QPOnBt=EstEaVNP+5t+m=FPNWshiPR3A@mail.gmail.com

Now, you might ask why not store this control information that we need
for cleanup purposes in the *main* shared memory segment rather than
in a dynamic shared memory segment. The basic problem is that I don't
know how to dig it out of there in any reasonable way. The dsm
control segment is small and has a very simple structure; when the
postmaster uses the previous postmaster's leftover control segment to
clean up orphaned shared memory segments, it will ignore that old
control segment unless it passes various sanity tests. But even if
passes those sanity tests but is corrupted somehow otherwise, nothing
that happens as a result will cause a fatal error, let alone a server
crash. You're of course welcome to critique that logic, but I tried
my best to make it bulletproof. See
dsm_cleanup_using_control_segment().

The structure of the main shared memory segment is way more
complicated. If we latched onto an old main shared memory segment,
we'd presumably need to traverse ShmemIndex to even find that portion
of the shared memory segment where the DSM control information was
slated to be stored. And there's no way that's going to be robust in
the face of a possibly-corrupted shared memory segment left over from
a previous run. And that's actually making the assumption that we
could even do it that way, which we really can't: as of 9.3, things
like ShmemIndex are stored in the MAP_SHARED anonymous mapping, and
the System V shared memory segment is small and fixed-size. We could
try to refactor the code so that we merge the control segment data
into the residual System V segment, but I think it'd be ugly and I'm
not sure what it really buys us.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company