Re: possible dsm bug in dsm_attach()

Hi,

dsm_attach() does the following:

nitems = dsm_control->nitems;
for (i = 0; i < nitems; ++i)
{
/* If the reference count is 0, the slot is actually unused. */
if (dsm_control->item[i].refcnt == 0)
continue;

/*
* If the reference count is 1, the slot is still in use, but the
* segment is in the process of going away. Treat that as if we
* didn't find a match.
*/
if (dsm_control->item[i].refcnt == 1)
break;

/* Otherwise, if the descriptor matches, we've found a match. */
if (dsm_control->item[i].handle == seg->handle)
{
dsm_control->item[i].refcnt++;
seg->control_slot = i;
break;
}
}

The break because of refcnt == 1 doesn't generally seem to be a good
idea. Why are we bailing if there's *any* segment that's in the process
of being removed? I think the check should be there *after* the
dsm_control->item[i].handle == seg->handle check?

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> dsm_attach() does the following:
>
> nitems = dsm_control->nitems;
> for (i = 0; i < nitems; ++i)
> {
> /* If the reference count is 0, the slot is actually unused. */
> if (dsm_control->item[i].refcnt == 0)
> continue;
>
> /*
> * If the reference count is 1, the slot is still in use, but the
> * segment is in the process of going away. Treat that as if we
> * didn't find a match.
> */
> if (dsm_control->item[i].refcnt == 1)
> break;
>
> /* Otherwise, if the descriptor matches, we've found a match. */
> if (dsm_control->item[i].handle == seg->handle)
> {
> dsm_control->item[i].refcnt++;
> seg->control_slot = i;
> break;
> }
> }
>
> The break because of refcnt == 1 doesn't generally seem to be a good
> idea. Why are we bailing if there's *any* segment that's in the process
> of being removed? I think the check should be there *after* the
> dsm_control->item[i].handle == seg->handle check?

You are correct. Good catch.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2014-05-06 08:48:57 -0400, Robert Haas wrote:
> On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > The break because of refcnt == 1 doesn't generally seem to be a good
> > idea. Why are we bailing if there's *any* segment that's in the process
> > of being removed? I think the check should be there *after* the
> > dsm_control->item[i].handle == seg->handle check?
>
> You are correct. Good catch.

Fix attached.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Tue, May 6, 2014 at 1:14 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2014-05-06 08:48:57 -0400, Robert Haas wrote:
>> On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> > The break because of refcnt == 1 doesn't generally seem to be a good
>> > idea. Why are we bailing if there's *any* segment that's in the process
>> > of being removed? I think the check should be there *after* the
>> > dsm_control->item[i].handle == seg->handle check?
>>
>> You are correct. Good catch.
>
> Fix attached.

Committed, thanks.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2014-05-06 13:45:13 -0400, Robert Haas wrote:
> On Tue, May 6, 2014 at 1:14 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > On 2014-05-06 08:48:57 -0400, Robert Haas wrote:
> >> On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> >> > The break because of refcnt == 1 doesn't generally seem to be a good
> >> > idea. Why are we bailing if there's *any* segment that's in the process
> >> > of being removed? I think the check should be there *after* the
> >> > dsm_control->item[i].handle == seg->handle check?
> >>
> >> You are correct. Good catch.
> >
> > Fix attached.
>
> Committed, thanks.

Heh. Not a fan of film references? :)

Thanks,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Tue, May 6, 2014 at 1:46 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
>> > Fix attached.
>>
>> Committed, thanks.
>
> Heh. Not a fan of film references? :)

I didn't quite put the pieces together there. I just thought the use
of "you" was awkward.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, May 6, 2014 at 11:15 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Tue, May 6, 2014 at 1:14 PM, Andres Freund <andres(at)2ndquadrant(dot)com>
wrote:
> > On 2014-05-06 08:48:57 -0400, Robert Haas wrote:
> >> On Tue, May 6, 2014 at 8:43 AM, Andres Freund <andres(at)2ndquadrant(dot)com>
wrote:
> >> > The break because of refcnt == 1 doesn't generally seem to be a good
> >> > idea. Why are we bailing if there's *any* segment that's in the
process
> >> > of being removed? I think the check should be there *after* the
> >> > dsm_control->item[i].handle == seg->handle check?
> >>
> >> You are correct. Good catch.
> >
> > Fix attached.
>
> Committed, thanks.
>

dsm_create(Size size, int flags)
{
..
/* Lock the control segment so we can register the new segment. */
LWLockAcquire(DynamicSharedMemoryControlLock, LW_EXCLUSIVE);

..
/* Verify that we can support an additional mapping. */
if (nitems >= dsm_control->maxitems)
{
if ((flags & DSM_CREATE_NULL_IF_MAXSEGMENTS) != 0)
{
dsm_impl_op(DSM_OP_DESTROY, seg->handle, 0, &seg->impl_private,
&seg->mapped_address, &seg->mapped_size, WARNING);
if (seg->resowner != NULL)
ResourceOwnerForgetDSM(seg->resowner, seg);
dlist_delete(&seg->node);
pfree(seg);
return NULL;
}
..
}

Is there a reason lock is not released in case we return NULL in above
code?

I am facing an issue in case we need to create many segments for
large inheritance hierarchy. Attached patch fixes the problem for me.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

On Wed, Mar 25, 2015 at 6:11 AM, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> I am facing an issue in case we need to create many segments for
> large inheritance hierarchy. Attached patch fixes the problem for me.

Sigh. You'd think I'd be able to write a 30-line patch without
introducing not one but two stupid bugs. But if you did think that,
you'd be wrong.

Committed.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company