Re: pg_class.relistemp

Lists: pgsql-hackers
From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: pg_class.relistemp
Date: 2011-07-13 17:28:42
Message-ID: BB8EF7E7-E364-4366-BEA2-586D46FAE398@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hackers,

With regard to this change:

http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=5f7b58fad8f45c69bb67944779dce67e2f481995

I'm wondering if it would be possible to restore the relistemp column to pg_class, at least for backwards compatibility, so that apps that expected it can continue to work on both 9.0 and 9.1. Even if it's read-only somehow, and the same as `relpersistence <> 't'`.

I've run into this with pgTAP, and am having a hard time coming up with a simple code path to support both without a patch. It'd make life simpler if there was some sort of compatibility interface so that my code doesn't have to maintain two paths.

Thanks,

David


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-13 19:14:51
Message-ID: 201107131914.p6DJEpQ03969@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

David E. Wheeler wrote:
> Hackers,
>
> With regard to this change:
>
> http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=5f7b58fad8f45c69bb67944779dce67e2f481995
>
> I'm wondering if it would be possible to restore the relistemp column
> to pg_class, at least for backwards compatibility, so that apps that
> expected it can continue to work on both 9.0 and 9.1. Even if it's
> read-only somehow, and the same as `relpersistence <> 't'`.

Uh, that is going to require an initdb, and it is unlinkely we are going
to need that this far into 9.1 beta. Also, we don't normally keep
system table columns around for backward compatibility because of the
confusion it can cause, e.g. which column do I look at?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-13 19:23:07
Message-ID: F0ADACAC-15A3-4E5A-A27E-6C9EE090589C@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 13, 2011, at 12:14 PM, Bruce Momjian wrote:

>> I'm wondering if it would be possible to restore the relistemp column
>> to pg_class, at least for backwards compatibility, so that apps that
>> expected it can continue to work on both 9.0 and 9.1. Even if it's
>> read-only somehow, and the same as `relpersistence <> 't'`.
>
> Uh, that is going to require an initdb, and it is unlinkely we are going
> to need that this far into 9.1 beta.

I was afraid of that.

> Also, we don't normally keep
> system table columns around for backward compatibility because of the
> confusion it can cause, e.g. which column do I look at?

The one that's documented.

Wasn't newsysviews supposed to deal with these sorts of issues? Why were they rejected?

Best,

David


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-13 19:24:35
Message-ID: 201107131924.p6DJOZp05149@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

David E. Wheeler wrote:
> On Jul 13, 2011, at 12:14 PM, Bruce Momjian wrote:
>
> >> I'm wondering if it would be possible to restore the relistemp column
> >> to pg_class, at least for backwards compatibility, so that apps that
> >> expected it can continue to work on both 9.0 and 9.1. Even if it's
> >> read-only somehow, and the same as `relpersistence <> 't'`.
> >
> > Uh, that is going to require an initdb, and it is unlinkely we are going
> > to need that this far into 9.1 beta.
>
> I was afraid of that.
>
> > Also, we don't normally keep
> > system table columns around for backward compatibility because of the
> > confusion it can cause, e.g. which column do I look at?
>
> The one that's documented.

Well, that assumes people read the documention and don't just do \d.
Keeping cruft around over time makes the system more complex.

> Wasn't newsysviews supposed to deal with these sorts of issues? Why
> were they rejected?

No idea.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +


From: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: David E(dot) Wheeler <david(at)kineticode(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-13 19:38:43
Message-ID: 1310585855-sup-388@alvh.no-ip.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Excerpts from Bruce Momjian's message of mié jul 13 15:24:35 -0400 2011:
> David E. Wheeler wrote:
> > On Jul 13, 2011, at 12:14 PM, Bruce Momjian wrote:
> >
> > >> I'm wondering if it would be possible to restore the relistemp column
> > >> to pg_class, at least for backwards compatibility, so that apps that
> > >> expected it can continue to work on both 9.0 and 9.1. Even if it's
> > >> read-only somehow, and the same as `relpersistence <> 't'`.
> > >
> > > Uh, that is going to require an initdb, and it is unlinkely we are going
> > > to need that this far into 9.1 beta.
> >
> > I was afraid of that.
> >
> > > Also, we don't normally keep
> > > system table columns around for backward compatibility because of the
> > > confusion it can cause, e.g. which column do I look at?
> >
> > The one that's documented.
>
> Well, that assumes people read the documention and don't just do \d.
> Keeping cruft around over time makes the system more complex.

This seems a case where column synonyms would have been useful (as was
the procpid / pid change).

--
Álvaro Herrera <alvherre(at)commandprompt(dot)com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-13 19:48:38
Message-ID: 020329B5-9709-4AE1-87E2-4029C93D4D33@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 13, 2011, at 12:38 PM, Alvaro Herrera wrote:

>> Well, that assumes people read the documention and don't just do \d.
>> Keeping cruft around over time makes the system more complex.
>
> This seems a case where column synonyms would have been useful (as was
> the procpid / pid change).

Well it couldn't just be that, because the data type has changed, too. Unless you could make a kind of "view column" or something where the expression was `relpersistence <> 't'`.

Best,

David


From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Alvaro Herrera" <alvherre(at)commandprompt(dot)com>, "David E(dot) Wheeler" <david(at)kineticode(dot)com>
Cc: "Bruce Momjian" <bruce(at)momjian(dot)us>, "Pg Hackers" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-13 19:57:41
Message-ID: 4E1DB265020000250003F2BC@gw.wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

"David E. Wheeler" <david(at)kineticode(dot)com> wrote:

> Unless you could make a kind of "view column" or something where the

> expression was `relpersistence <> 't'`.

create or replace function relistemp(rel pg_class)
returns boolean language sql immutable strict as
$$select $1.relpersistence = 't';$$;

Just don't forget to use the table name or alias in front of it... :-)

-Kevin


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>
Cc: "Alvaro Herrera" <alvherre(at)commandprompt(dot)com>, "Bruce Momjian" <bruce(at)momjian(dot)us>, "Pg Hackers" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 16:51:10
Message-ID: C57198A6-E8D8-4711-B821-8984A733EF47@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 13, 2011, at 12:57 PM, Kevin Grittner wrote:

> create or replace function relistemp(rel pg_class)
> returns boolean language sql immutable strict as
> $$select $1.relpersistence = 't';$$;
>
> Just don't forget to use the table name or alias in front of it... :-)

Oh, nice hack. How far back does that work (pgTAP runs on 8.0 and higher)?

Thanks,

David


From: Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
To: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
Cc: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>, Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 16:55:22
Message-ID: 4E1F1F7A.90100@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 14.07.2011 19:51, David E. Wheeler wrote:
> On Jul 13, 2011, at 12:57 PM, Kevin Grittner wrote:
>
>> create or replace function relistemp(rel pg_class)
>> returns boolean language sql immutable strict as
>> $$select $1.relpersistence = 't';$$;
>>
>> Just don't forget to use the table name or alias in front of it... :-)
>
> Oh, nice hack. How far back does that work (pgTAP runs on 8.0 and higher)?

Far back. But you only need it in >= 9.1. Older versions have the
pg_class.relistemp column anyway.

Not sure how this helps, though. If you modify pgTAP to install that
automatically in pgTAP when dealing with a new server version, you might
as well modify its queries to use relispersistence = 't' directly when
dealing with a new server version. It works as a manual work-around if
you can't upgrade pgTAP, I guess.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com


From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
Cc: "Alvaro Herrera" <alvherre(at)commandprompt(dot)com>, "Bruce Momjian" <bruce(at)momjian(dot)us>, "Pg Hackers" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 16:57:03
Message-ID: 4E1ED98F020000250003F331@gw.wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

"David E. Wheeler" <david(at)kineticode(dot)com> wrote:
> On Jul 13, 2011, at 12:57 PM, Kevin Grittner wrote:
>
>> create or replace function relistemp(rel pg_class)
>> returns boolean language sql immutable strict as
>> $$select $1.relpersistence = 't';$$;
>>
>> Just don't forget to use the table name or alias in front of
>> it... :-)
>
> Oh, nice hack. How far back does that work (pgTAP runs on 8.0 and
> higher)?

As far as I know, the technique of creating a function with a record
type as its only parameter to use as a "generated column" goes way
back. This particular function won't work prior to 9.1, because you
won't have the relpersistence column, but then, prior to 9.1 you
wouldn't need to run this because you already have a relistemp
column.

-Kevin


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
Cc: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>, Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 16:59:05
Message-ID: 16053D45-19A2-4650-8561-225B103B8643@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 14, 2011, at 9:55 AM, Heikki Linnakangas wrote:

> Far back. But you only need it in >= 9.1. Older versions have the pg_class.relistemp column anyway.

Yeah.

> Not sure how this helps, though. If you modify pgTAP to install that automatically in pgTAP when dealing with a new server version, you might as well modify its queries to use relispersistence = 't' directly when dealing with a new server version. It works as a manual work-around if you can't upgrade pgTAP, I guess.

Yeah, that's what I'd rather avoid. I'll probably have to modify the function that makes the call to look at the version number. Annoying, but do-able.

https://github.com/theory/pgtap/blob/master/sql/pgtap.sql.in#L5894

Best,

David


From: Josh Berkus <josh(at)agliodbs(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_class.relistemp
Date: 2011-07-14 17:52:09
Message-ID: 4E1F2CC9.9070008@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

All,

BTW, if we're dumping relistemp, we're going to need to notify every
maker of a PostgreSQL admin interface before we release 9.1.

This is why we should have had a complete set of sysviews ...

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_class.relistemp
Date: 2011-07-14 17:53:15
Message-ID: 201107141753.p6EHrFJ12660@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Josh Berkus wrote:
> All,
>
> BTW, if we're dumping relistemp, we're going to need to notify every
> maker of a PostgreSQL admin interface before we release 9.1.
>
> This is why we should have had a complete set of sysviews ...

Are they not testing our 9.1 betas?

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_class.relistemp
Date: 2011-07-14 19:19:59
Message-ID: 7903.1310671199@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Bruce Momjian <bruce(at)momjian(dot)us> writes:
> Josh Berkus wrote:
>> BTW, if we're dumping relistemp, we're going to need to notify every
>> maker of a PostgreSQL admin interface before we release 9.1.

> Are they not testing our 9.1 betas?

There has never, ever, been a guarantee that the system catalogs don't
change across versions. Anybody issuing such queries should expect to
have to retest them and possibly change them in each new major release.
I see nothing to sweat about here.

regards, tom lane


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, Josh Berkus <josh(at)agliodbs(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_class.relistemp
Date: 2011-07-14 20:10:53
Message-ID: B9D780B2-7B67-4E2C-A5C6-07DAD1AE0938@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 14, 2011, at 12:19 PM, Tom Lane wrote:

>> Are they not testing our 9.1 betas?
>
> There has never, ever, been a guarantee that the system catalogs don't
> change across versions. Anybody issuing such queries should expect to
> have to retest them and possibly change them in each new major release.
> I see nothing to sweat about here.

A deprecation cycle at least might be useful.

Best,

David


From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "David E(dot) Wheeler" <david(at)kineticode(dot)com>, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Josh Berkus" <josh(at)agliodbs(dot)com>, "Bruce Momjian" <bruce(at)momjian(dot)us>, <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 20:26:13
Message-ID: 4E1F0A95020000250003F349@gw.wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

"David E. Wheeler" <david(at)kineticode(dot)com> wrote:

> A deprecation cycle at least might be useful.

How about a "relistemp" extension on pgxn.org for the "generated
column" function to provide the backward compatibility? Is the new
extension mechanism a sane way to help those who need a phase-out
period?

-Kevin


From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_class.relistemp
Date: 2011-07-14 20:59:51
Message-ID: 4E1F58C7.4010305@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


> There has never, ever, been a guarantee that the system catalogs don't
> change across versions. Anybody issuing such queries should expect to
> have to retest them and possibly change them in each new major release.

I know that's always been our policy. It his, however,
vendor-unfriendly because we don't supply any interface for many things
(such as temp tables) other than the system catalogs.

So if we're going to break compatibility, then we could stand to make a
little noise about it.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Bruce Momjian <bruce(at)momjian(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_class.relistemp
Date: 2011-07-14 21:06:33
Message-ID: CABUevEyK8Z=mTxCh5WejcZnBUUPHPuebcO_jt6Q2gWH_TrAY3g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Thu, Jul 14, 2011 at 21:59, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
>
>> There has never, ever, been a guarantee that the system catalogs don't
>> change across versions.  Anybody issuing such queries should expect to
>> have to retest them and possibly change them in each new major release.
>
> I know that's always been our policy.  It his, however,
> vendor-unfriendly because we don't supply any interface for many things
> (such as temp tables) other than the system catalogs.
>
> So if we're going to break compatibility, then we could stand to make a
> little noise about it.

We've broken the admin apps in pretty much every single release. And
they generally don't complain. If someone developing an admin app
hasn't been doing extensive testing starting *at the latest* with
beta1 (and recommended per each alpha), they shouldn't expect to
release until quite long after the release.

That said, a stable set of system views certainly wouldn't hurt - but
making extra noise about a simple change like this one would likely
not make a difference.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


From: Dave Page <dpage(at)pgadmin(dot)org>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 21:10:19
Message-ID: CA+OCxowVih-Lu7mym1toUY3nrNrxBrjiVRVavAUpCqrd1bdiCw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Thursday, July 14, 2011, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
>
>> There has never, ever, been a guarantee that the system catalogs don't
>> change across versions.  Anybody issuing such queries should expect to
>> have to retest them and possibly change them in each new major release.
>
> I know that's always been our policy.  It his, however,
> vendor-unfriendly because we don't supply any interface for many things
> (such as temp tables) other than the system catalogs.
>
> So if we're going to break compatibility, then we could stand to make a
> little noise about it.

As one of said vendors, I completely disagree. There are a ton of
things that change with each release, and all we do by putting in
hacks for backwards compatibility is add bloat that needs to be
maintained, and encourage vendors to be lazy.

Break compatibility is actually something that is important to us - it
forces us to fix obvious issues, and makes it much harder to
inadvertently miss important changes.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Dave Page <dpage(at)pgadmin(dot)org>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 21:48:36
Message-ID: 59B8FFAE-B6DD-45C8-90D0-79BDFAF0299C@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 14, 2011, at 2:10 PM, Dave Page wrote:

> Break compatibility is actually something that is important to us - it
> forces us to fix obvious issues, and makes it much harder to
> inadvertently miss important changes.

Agreed, but a deprecation cycle would be much appreciated.

David


From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Dave Page <dpage(at)pgadmin(dot)org>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 21:54:20
Message-ID: 4E1F658C.5060402@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


> As one of said vendors, I completely disagree.

I don't agree that you qualify as a vendor. You're on the friggin' core
team.

I'm talking about vendors like DBVizualizer or TORA, for which
PostgreSQL is just one of the databases they support. If stuff breaks
gratuitously, the reaction of some of them is always to either drop
support or delay it for a year or more. This doesn't benefit our community.

> There are a ton of
> things that change with each release, and all we do by putting in
> hacks for backwards compatibility is add bloat that needs to be
> maintained, and encourage vendors to be lazy.

I don't agree that having comprehensive system views with multi-version
stability would be a "hack".

> Break compatibility is actually something that is important to us - it
> forces us to fix obvious issues, and makes it much harder to
> inadvertently miss important changes.

What I'm hearing from you is: "Breaking backwards compatibility is
something we should do more of because it lets us know which vendors are
paying attention and weeds out the unfit." Is that what you meant to say?

That seems like a way to ensure that PostgreSQL support continue to be
considered optional, or based on outdated versions, for multi-database
tools.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: Dave Page <dpage(at)pgadmin(dot)org>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 22:05:00
Message-ID: 13901.1310681100@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Josh Berkus <josh(at)agliodbs(dot)com> writes:
>> There are a ton of
>> things that change with each release, and all we do by putting in
>> hacks for backwards compatibility is add bloat that needs to be
>> maintained, and encourage vendors to be lazy.

> I don't agree that having comprehensive system views with multi-version
> stability would be a "hack".

If we had that, it wouldn't be a hack. Putting in a hack to cover the
specific case of relistemp, on the other hand, is just a hack.

The real question here, IMO, is "how many applications are there that
really need to know about temporary relations, but have no interest in
the related feature of unlogged relations?". Because only such apps
would be served by a compatibility hack for this. An app that thinks it
knows the semantics of relistemp, and isn't updated to grok unlogged
tables, may be worse than broken --- it may be silently incorrect.

regards, tom lane


From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Dave Page <dpage(at)pgadmin(dot)org>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 22:06:25
Message-ID: 4E1F6861.3080203@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


>> I don't agree that having comprehensive system views with multi-version
>> stability would be a "hack".
>
> If we had that, it wouldn't be a hack. Putting in a hack to cover the
> specific case of relistemp, on the other hand, is just a hack.

I agree.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Dave Page <dpage(at)pgadmin(dot)org>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-14 22:13:47
Message-ID: 4716E21C-8E11-4379-AA3D-63E9B3E47FC5@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 14, 2011, at 3:05 PM, Tom Lane wrote:

> Josh Berkus <josh(at)agliodbs(dot)com> writes:
>>> There are a ton of
>>> things that change with each release, and all we do by putting in
>>> hacks for backwards compatibility is add bloat that needs to be
>>> maintained, and encourage vendors to be lazy.
>
>> I don't agree that having comprehensive system views with multi-version
>> stability would be a "hack".
>
> If we had that, it wouldn't be a hack.

Is that an endorsement for adding such a feature?

> Putting in a hack to cover the
> specific case of relistemp, on the other hand, is just a hack.

Sure.

> The real question here, IMO, is "how many applications are there that
> really need to know about temporary relations, but have no interest in
> the related feature of unlogged relations?". Because only such apps
> would be served by a compatibility hack for this. An app that thinks it
> knows the semantics of relistemp, and isn't updated to grok unlogged
> tables, may be worse than broken --- it may be silently incorrect.

So pgTAP creates temporary tables to store result sets so that it can then compare the results of two queries. The function in question was getting a list of columns in such a temporary table in order to make sure that the types were the same between two such tables before comparing results. It checked relistemp to make sure it was looking at the temp table rather than some other table that might happen to have the same name.

So now the query looks like this:

SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
FROM pg_catalog.pg_attribute a
JOIN pg_catalog.pg_class c ON a.attrelid = c.oid
WHERE c.relname = $1
-- AND c.relistemp -- 8.3-9.0
AND c.relpersistence = 't' -- 9.1
AND attnum > 0
AND NOT attisdropped
ORDER BY attnum

Is that all I need to do, or is there something else I should be aware of with regard to unlogged tables?

Thanks,

David


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Josh Berkus <josh(at)agliodbs(dot)com>, Dave Page <dpage(at)pgadmin(dot)org>, BruceMomjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-15 01:43:30
Message-ID: C3025505-452B-4F00-B697-A2890579EF45@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 14, 2011, at 5:13 PM, "David E. Wheeler" <david(at)kineticode(dot)com> wrote:
> On Jul 14, 2011, at 3:05 PM, Tom Lane wrote:
>
>> Josh Berkus <josh(at)agliodbs(dot)com> writes:
>>>> There are a ton of
>>>> things that change with each release, and all we do by putting in
>>>> hacks for backwards compatibility is add bloat that needs to be
>>>> maintained, and encourage vendors to be lazy.
>>
>>> I don't agree that having comprehensive system views with multi-version
>>> stability would be a "hack".
>>
>> If we had that, it wouldn't be a hack.
>
> Is that an endorsement for adding such a feature?
>
>> Putting in a hack to cover the
>> specific case of relistemp, on the other hand, is just a hack.
>
> Sure.
>
>> The real question here, IMO, is "how many applications are there that
>> really need to know about temporary relations, but have no interest in
>> the related feature of unlogged relations?". Because only such apps
>> would be served by a compatibility hack for this. An app that thinks it
>> knows the semantics of relistemp, and isn't updated to grok unlogged
>> tables, may be worse than broken --- it may be silently incorrect.
>
> So pgTAP creates temporary tables to store result sets so that it can then compare the results of two queries. The function in question was getting a list of columns in such a temporary table in order to make sure that the types were the same between two such tables before comparing results. It checked relistemp to make sure it was looking at the temp table rather than some other table that might happen to have the same name.
>
> So now the query looks like this:
>
> SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
> FROM pg_catalog.pg_attribute a
> JOIN pg_catalog.pg_class c ON a.attrelid = c.oid
> WHERE c.relname = $1
> -- AND c.relistemp -- 8.3-9.0
> AND c.relpersistence = 't' -- 9.1
> AND attnum > 0
> AND NOT attisdropped
> ORDER BY attnum
>
> Is that all I need to do, or is there something else I should be aware of with regard to unlogged tables?

Probably not, in this case. Just a thought: maybe you could rewrite the query to check whether the namespace name starts with pg_temp. Maybe that would be version-independent...

...Robert


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Josh Berkus <josh(at)agliodbs(dot)com>, Dave Page <dpage(at)pgadmin(dot)org>, BruceMomjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-15 01:58:38
Message-ID: BAB29720-8CBC-4EBC-9E24-98E17F334020@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 14, 2011, at 6:43 PM, Robert Haas wrote:

>> Is that all I need to do, or is there something else I should be aware of with regard to unlogged tables?
>
> Probably not, in this case. Just a thought: maybe you could rewrite the query to check whether the namespace name starts with pg_temp. Maybe that would be version-independent...

Ah, good idea, I forgot about pg_temp.

David


From: Dave Page <dpage(at)pgadmin(dot)org>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-15 08:09:18
Message-ID: CA+OCxozF-9AC9M0qYaNHTY_RtriqXNrv_MzeZCRn3ikhFnrUbw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Thu, Jul 14, 2011 at 10:54 PM, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
>
>> As one of said vendors, I completely disagree.
>
> I don't agree that you qualify as a vendor.  You're on the friggin' core
> team.

And I look after the development of the leading open source management
tool for PostgreSQL, as well as a number of tools at EnterpriseDB. I
might be on the core team, but I still build tools.

> I'm talking about vendors like DBVizualizer or TORA, for which
> PostgreSQL is just one of the databases they support.  If stuff breaks
> gratuitously, the reaction of some of them is always to either drop
> support or delay it for a year or more.  This doesn't benefit our community.

I'm not talking about gratuitously breaking things - just not masking
essential changes.

>> There are a ton of
>> things that change with each release, and all we do by putting in
>> hacks for backwards compatibility is add bloat that needs to be
>> maintained, and encourage vendors to be lazy.
>
> I don't agree that having comprehensive system views with multi-version
> stability would be a "hack".

That isn't what was being suggested.

>> Break compatibility is actually something that is important to us - it
>> forces us to fix obvious issues, and makes it much harder to
>> inadvertently miss important changes.
>
> What I'm hearing from you is: "Breaking backwards compatibility is
> something we should do more of because it lets us know which vendors are
> paying attention and weeds out the unfit."   Is that what you meant to say?

No, I meant to say precisely what I did say. By not masking changes in
the catalogs, we draw attention to things that have changed for good
reason, and almost certainly need to be addressed.

> That seems like a way to ensure that PostgreSQL support continue to be
> considered optional, or based on outdated versions, for multi-database
> tools.

Whilst this particular case might be safe to just ignore in third part
tools, other changes to the catalogs are not, and masking them could
potentially hide bugs or issues that need to be fixed to actually work
properly with the newer version of the server.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_class.relistemp
Date: 2011-07-15 16:25:36
Message-ID: 2259.1310747136@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Thu, Jul 14, 2011 at 21:59, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
>> So if we're going to break compatibility, then we could stand to make a
>> little noise about it.

> We've broken the admin apps in pretty much every single release. And
> they generally don't complain.

Yeah. Quite honestly, this thread is trying to turn a molehill into a
mountain. I will confidently predict that the really big, nasty change
in 9.1 is the change of default standard_conforming_strings. That's
going to break way more apps than anything else, and possibly in
security-critical ways. Anybody who moves an app onto 9.1 without
testing it is going to suffer big-time.

regards, tom lane


From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: pg_class.relistemp
Date: 2011-07-15 16:36:00
Message-ID: CABUevExcHssSb_85hZ-CdbESW39kWNnt7HvFw1P0HFc9Z70MHQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 15, 2011 at 17:25, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Thu, Jul 14, 2011 at 21:59, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
>>> So if we're going to break compatibility, then we could stand to make a
>>> little noise about it.
>
>> We've broken the admin apps in pretty much every single release. And
>> they generally don't complain.
>
> Yeah.  Quite honestly, this thread is trying to turn a molehill into a
> mountain.  I will confidently predict that the really big, nasty change
> in 9.1 is the change of default standard_conforming_strings.  That's
> going to break way more apps than anything else, and possibly in
> security-critical ways.  Anybody who moves an app onto 9.1 without
> testing it is going to suffer big-time.

+(a lot)

in fact, I think we should definitely "shout out" more clearly about
that one in the release notes. Yes, it's the very first item. But I
think it deserves a big fat warning box as well.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Dave Page <dpage(at)pgadmin(dot)org>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-15 16:41:53
Message-ID: 2601.1310748113@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

"David E. Wheeler" <david(at)kineticode(dot)com> writes:
> So pgTAP creates temporary tables to store result sets so that it can then compare the results of two queries. The function in question was getting a list of columns in such a temporary table in order to make sure that the types were the same between two such tables before comparing results. It checked relistemp to make sure it was looking at the temp table rather than some other table that might happen to have the same name.

Well, actually, that code flat out doesn't work, so whether relistemp is
available in 9.1 is the least of your problems. Consider what would
happen if two concurrent sessions did this with the same temp table
name.

How about doing this instead?

SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
FROM pg_catalog.pg_attribute a
JOIN pg_catalog.pg_class c ON a.attrelid = c.oid
WHERE c.oid = 'pg_temp.tablenamehere'::pg_catalog.regclass
AND attnum > 0
AND NOT attisdropped
ORDER BY attnum

This would only work in releases that know the pg_temp abbreviation,
which includes any minor release later than March 2007. But since
relistemp doesn't even exist before 8.4 (released in 2009), that's still
more backwards-portable than what you've got. You could also just do
'tablenamehere'::pg_catalog.regclass and trust that the user didn't move
pg_temp to the back of the search path.

regards, tom lane


From: Jim Nasby <jim(at)nasby(dot)net>
To: David E(dot) Wheeler <david(at)kineticode(dot)com>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-16 00:17:42
Message-ID: 8DD9ED4C-342F-4E83-881C-B874AD12692B@nasby.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 13, 2011, at 2:23 PM, David E. Wheeler wrote:
> Wasn't newsysviews supposed to deal with these sorts of issues? Why were they rejected?

Unless they recently came up again and got rejected again; the original complaint was that some of their conventions didn't follow information_schema conventions. The community wanted that changed and that never happened.
--
Jim C. Nasby, Database Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Jim Nasby <jim(at)nasby(dot)net>
Cc: "David E(dot) Wheeler" <david(at)kineticode(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-17 02:16:38
Message-ID: CA+TgmoZXgeenOLs888pt8YGHhv=xUaAD=DyRc-jWAALFzM3xTw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 15, 2011 at 8:17 PM, Jim Nasby <jim(at)nasby(dot)net> wrote:
> On Jul 13, 2011, at 2:23 PM, David E. Wheeler wrote:
>> Wasn't newsysviews supposed to deal with these sorts of issues? Why were they rejected?
>
> Unless they recently came up again and got rejected again; the original complaint was that some of their conventions didn't follow information_schema conventions. The community wanted that changed and that never happened.

I think, also, that the idea that newsysviews is going to fix all of
our problems is mostly wishful thinking. Let's suppose that we had a
system view over pg_class that kept around some variant of relistemp
even though it's gone from pg_class per se. Well, such a column would
probably be false for both unlogged and permanent tables and true for
temporary tables and David would be happy.

But what happens when and if we add global temporary tables? Now we
might very well decide to set the faux-relistemp to true for temporary
and global temporary tables (they do have "temporary" in the name,
after all!) and false for unlogged and permanent tables. Or we might
decide that the faux-relistemp should only be true for the kind of
temporary tables that we've always had, and false for these new global
temporary tables, perhaps on the theory that a global temporary table
is not really temporary at all, though its contents are. One of these
decisions would probably be right for David (and pgTap) and the other
would be wrong; and the decision that was right for pgTap might be
wrong for some other client. So instead of breaking pgTap we might
just quietly make it stop working correctly.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Jim Nasby <jim(at)nasby(dot)net>, Bruce Momjian <bruce(at)momjian(dot)us>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-17 02:31:39
Message-ID: 96E7B156-D8A4-429E-8D27-E226AE210DCD@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 16, 2011, at 7:16 PM, Robert Haas wrote:

> But what happens when and if we add global temporary tables? Now we
> might very well decide to set the faux-relistemp to true for temporary
> and global temporary tables (they do have "temporary" in the name,
> after all!) and false for unlogged and permanent tables. Or we might
> decide that the faux-relistemp should only be true for the kind of
> temporary tables that we've always had, and false for these new global
> temporary tables, perhaps on the theory that a global temporary table
> is not really temporary at all, though its contents are. One of these
> decisions would probably be right for David (and pgTap) and the other
> would be wrong; and the decision that was right for pgTap might be
> wrong for some other client. So instead of breaking pgTap we might
> just quietly make it stop working correctly.

Well I think it would continue to work exactly as it has in the past. And if one needed to know other information about the *type* of temp table, well then one would have to use relpersistence.

The idea is not to try to make it adapt to future changes. The idea is to try to preserve the previous behavior for some period of time.

Best,

David


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, Dave Page <dpage(at)pgadmin(dot)org>, Bruce Momjian <bruce(at)momjian(dot)us>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_class.relistemp
Date: 2011-07-22 17:34:39
Message-ID: DE0DDEC7-2A01-4481-8332-144D7E748DC3@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Jul 15, 2011, at 9:41 AM, Tom Lane wrote:

> Well, actually, that code flat out doesn't work, so whether relistemp is
> available in 9.1 is the least of your problems. Consider what would
> happen if two concurrent sessions did this with the same temp table
> name.

Oh. Duh.

> How about doing this instead?
>
> SELECT pg_catalog.format_type(a.atttypid, a.atttypmod)
> FROM pg_catalog.pg_attribute a
> JOIN pg_catalog.pg_class c ON a.attrelid = c.oid
> WHERE c.oid = 'pg_temp.tablenamehere'::pg_catalog.regclass
> AND attnum > 0
> AND NOT attisdropped
> ORDER BY attnum

I always forget that "$schema.$tablename"::regclass will work.

> This would only work in releases that know the pg_temp abbreviation,
> which includes any minor release later than March 2007. But since
> relistemp doesn't even exist before 8.4 (released in 2009), that's still
> more backwards-portable than what you've got. You could also just do
> 'tablenamehere'::pg_catalog.regclass and trust that the user didn't move
> pg_temp to the back of the search path.

Yeah, this is a much better solution. Many thanks, Tom, just what I needed.

Best,

David