Re: Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

Lists: pgsql-hackers
From: Nikhil Sontakke <nikkhils(at)gmail(dot)com>
To:
Cc: Daniel Farina <daniel(at)heroku(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers
Date: 2011-11-11 07:57:32
Message-ID: CANgU5ZcfF82nwzjTiov8S2=_H_kTS_qQ0YH09ep3bneRY7yiAQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi,

> But if it's deemed to be a
> problem, I want to see a solution that's actually watertight.)
>
>
After Daniel's hunch about pg_dump barfing due to such leftover entries
proving out to be true, we have one credible explanation (there might be
other reasons too) for this long standing issue. I see some reports from as
early as 2004 and some as latest as Feb, 2011!

http://archives.postgresql.org/pgsql-bugs/2011-02/msg00185.php

One way in 9.x could be to modify get_object_address to additionally accept
objoid as an argument and use that to lock the schema in AccessShareLock
mode from all places where schema based objects (tables, views, types,
sequences, functions, indexes, extensions, constraints, operators stuff, ts
stuff, rules, domains, etc. phew!) can be created. Or since this is schema
specific, we can as well right a new function to do this. We might also add
logic to only lock user created schemas.

This can be added right after the namespace for the involved object has
been correctly identified. The lock can then get released later as part of
the transaction commit.

Regards,
Nikhils


From: Daniel Farina <daniel(at)heroku(dot)com>
To: Nikhil Sontakke <nikkhils(at)gmail(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers
Date: 2011-11-11 19:08:01
Message-ID: CAAZKuFb2d1EqgLY7BV3d1hYW613q2FUOrHuqXVR12a8sT=6waQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Thu, Nov 10, 2011 at 11:57 PM, Nikhil Sontakke <nikkhils(at)gmail(dot)com> wrote:
> Hi,
>
>>
>> But if it's deemed to be a
>> problem, I want to see a solution that's actually watertight.)
>>
>
> After Daniel's hunch about pg_dump barfing due to such leftover entries
> proving out to be true, we have one credible explanation (there might be
> other reasons too) for this long standing issue. I see some reports from as
> early as 2004 and some as latest as Feb, 2011!

The 2004 report was in version 7.3, released in 2002. So it's Very
Nearly a ten year old bug, and may very well go back further back in
time.

--
fdr


From: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
To: Daniel Farina <daniel(at)heroku(dot)com>
Cc: Nikhil Sontakke <nikkhils(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers
Date: 2011-11-11 19:21:31
Message-ID: 1321039140-sup-4806@alvh.no-ip.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


Excerpts from Daniel Farina's message of vie nov 11 16:08:01 -0300 2011:
> On Thu, Nov 10, 2011 at 11:57 PM, Nikhil Sontakke <nikkhils(at)gmail(dot)com> wrote:
> > Hi,
> >
> >>
> >> But if it's deemed to be a
> >> problem, I want to see a solution that's actually watertight.)
> >>
> >
> > After Daniel's hunch about pg_dump barfing due to such leftover entries
> > proving out to be true, we have one credible explanation (there might be
> > other reasons too) for this long standing issue. I see some reports from as
> > early as 2004 and some as latest as Feb, 2011!
>
> The 2004 report was in version 7.3, released in 2002. So it's Very
> Nearly a ten year old bug, and may very well go back further back in
> time.

Wasn't 7.3 the release that introduced schemas in the first place? I
wonder if there's any other kind of "container" sort of object that
could present a similar problem, in releases prior to that.

If we delay fixing it for 16 more days, it would last nine years.

--
Álvaro Herrera <alvherre(at)commandprompt(dot)com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Cc: Daniel Farina <daniel(at)heroku(dot)com>, Nikhil Sontakke <nikkhils(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers
Date: 2011-11-11 19:25:58
Message-ID: 9453.1321039558@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> Excerpts from Daniel Farina's message of vie nov 11 16:08:01 -0300 2011:
>> The 2004 report was in version 7.3, released in 2002. So it's Very
>> Nearly a ten year old bug, and may very well go back further back in
>> time.

> Wasn't 7.3 the release that introduced schemas in the first place?

I think there's a very good chance that the older reports with similar
symptoms are completely unrelated, anyhow.

regards, tom lane


From: Nikhil Sontakke <nikkhils(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Daniel Farina <daniel(at)heroku(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers
Date: 2011-11-12 04:55:12
Message-ID: CANgU5ZfpnmJC+9z9LZirojBKb2x1iLYgX8jFK+w7uXStXjZWKQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

> > Wasn't 7.3 the release that introduced schemas in the first place?
>
> I think there's a very good chance that the older reports with similar
> symptoms are completely unrelated, anyhow.
>
>
Tom Lane is reluctant and that should tell me something :)

So unless the list feels that this should be fixed and also agrees on the
general approach, I will not touch a single line of code. Obviously someone
else is welcome to have a stab at this too.

Regards,
Nikhils