Re: Can we revisit the thought of PostgreSQL 7.2.4?

Hi everyone,

Over the last few days we've had patches submitted for 7.2.3 that
address a couple of things, both the WAL Recovery Bug that Tom has
developed a patch for, and a couple of buffer overflows that have been
widely reported.

Although we haven't wanted to release a 7.2.4, and have instead
encouraged people to upgrade to 7.3.x, there are places out there who's
applications aren't compatible with 7.3.x and would also need to upgrade
them as well.

It might be a really good idea if we re-visit the thought of 7.2.4 and
have something that people running the 7.2.x series can use safely until
they are able to move to 7.3.x or above.

What would it take, and apart from patches for the buffer overflows and
the WAL recovery bug, should anything else be included to ensure safety
and stability?

:-)

Regards and best wishes,

Justin Clift

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

On Thursday 16 January 2003 22:47, Justin Clift wrote:
> Although we haven't wanted to release a 7.2.4, and have instead
> encouraged people to upgrade to 7.3.x, there are places out there who's
> applications aren't compatible with 7.3.x and would also need to upgrade
> them as well.

Incidentally, has anyone else noticed the security update onslaught from Red
Hat for older PostgreSQL versions? They even backported the fixes to 6.5.3
from Red Hat 6.2 (as well as for 7.0 and 7.1 as released in the respective
Red Hat Linux versions). Should I forward that notice here?
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar(dot)owen(at)wgcr(dot)org> writes:
> Incidentally, has anyone else noticed the security update onslaught from Red
> Hat for older PostgreSQL versions? They even backported the fixes to 6.5.3
> from Red Hat 6.2 (as well as for 7.0 and 7.1 as released in the respective
> Red Hat Linux versions). Should I forward that notice here?

Some of the guys in Toronto got excited about it, but I can't see a lot
of value there myself. If you're still running 6.5.3, is it likely you
notice updates from anywhere?

Red Hat 6.2 is still nominally supported (until March 31, it says here)
so I suppose there's a corporate compulsion to back-patch anything
that's labeled a security issue. But let's get real ... PG 6.anything
is stone-age code now.

regards, tom lane
Red Hat Database project

PS: I'm not taking a position on Justin's suggestion that there should
be a 7.2.4. Marc and Bruce would be the ones who have to do the work,
so they get to make the decision...

Tom Lane wrote:
> Red Hat 6.2 is still nominally supported (until March 31, it says here)
> so I suppose there's a corporate compulsion to back-patch anything
> that's labeled a security issue. But let's get real ... PG 6.anything
> is stone-age code now.
>
> regards, tom lane
> Red Hat Database project
>
> PS: I'm not taking a position on Justin's suggestion that there should
> be a 7.2.4. Marc and Bruce would be the ones who have to do the work,
> so they get to make the decision...

Who, us? Well, there is the confusion factor of releasing a patch to a
superceeded major version. Wrapping it up and putting it out really
isn't a big deal. Marc?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Saturday 18 January 2003 00:08, Tom Lane wrote:
> Lamar Owen <lamar(dot)owen(at)wgcr(dot)org> writes:
> > Incidentally, has anyone else noticed the security update onslaught from
> > Red Hat for older PostgreSQL versions? They even backported the fixes to
> > 6.5.3 from Red Hat 6.2 (as well as for 7.0 and 7.1 as released in the
> > respective Red Hat Linux versions). Should I forward that notice here?

> Some of the guys in Toronto got excited about it, but I can't see a lot
> of value there myself. If you're still running 6.5.3, is it likely you
> notice updates from anywhere?

Why not? Upgrading to another major version of most things isn't supported by
Red Hat within a particular version. KDE is a prime example. GNOME is
another. XFree86 is another. BIND is yet another, although BIND8 upgrades
are available for the systems that shipped with BIND4. RPM itself is one of
the few exceptions. Going to 7.3 from 6.5 is not an update.

And lots of sites are still running 6.2. IIRC Red Hat's up2date automatic
upgrader tool was available in 6.2, and I know other autoupdaters are
available. And, as we all know, automatic upgrade of PostgreSQL is only
possible within a major version. Plenty of security conscious people still
run Red Hat 6.2. Many probably pay for the enterprise support contracts
through Red Hat, costing much money.

Hmmmph, I know of a user running a couple of sites still running 5.2, with no
intention of upgrading those machines. Will PostgreSQL 7.3 even build on Red
Hat Linux 5.2? Forced upgrades are nonsense. So I'm glad Red Hat decided to
put resources into supporting their userbase (even given the sunset on said
support).

On the BSD front, OpenBSD in particular still is running ancient versions of
some core network stuff, due to the extreme security nature of that OS. Last
I looked at OBSD it still shipped BIND4. 4.9 something. Positively ancient
code that they have thoroughly audited. BIND8 hadn't at that time been fully
audited.

> Red Hat 6.2 is still nominally supported (until March 31, it says here)
> so I suppose there's a corporate compulsion to back-patch anything
> that's labeled a security issue. But let's get real ... PG 6.anything
> is stone-age code now.

Why? If a user doesn't need the features of 7.x.x, and the codebase is
working well for him/her, why should said user/DBA feel compelled to go
through who knows what mechanations to upgrade to the latest version? That's
Microsoft-think. The upgrade from a 6.5.3 system to a 7.3.1 system is likely
to be traumatic at least and cataclysmic at worst (to upgrade PostgreSQL may
require upgrading the whole OS, which may require more memory (maybe more
memory than the motherboard will support, even)....). Yes, let's get real --
not everybody needs or necessarily even wants all the improved features of PG
7.3 versus even 6.5.

The 'corporate compulsion' you mentioned is more widely known as 'customer
service.' IOW, you want to stay in business, you support your customers.

The Red Hat 5.2 user mentioned previously is perfectly happy with the
featureset of PostgreSQL 6.3.2 (which is what 5.2 shipped with) and won't
upgrade until it's very necessary. But this is a very low resource machine,
where even the Linux 2.0 kernel makes sense. Now 6.5.3 will build on 5.2,
but I haven't tried anything more recent. And 6.3.2 is enough database for
their uses -- but these machines are in roles where security issues could be
problematic. If it were easier to upgrade, they might consider it.

_Of_course_ I'm not advocating that _we_ support these old of systems (after
all, the PostgreSQL Global Development Group _has_ no customers) -- but it
_is_ nice when a distributor acknowledges their older customers with real
security updates within their released versions, and doesn't force major
upgrades when unnecessary.

Now if the user needs _features_ then the upgrade is justified, and I have no
sympathy for a user who wants, say, schema support backpatched to 7.0.3, for
instance. That request is just ridiculous. But for security and critical
bugfixes, it should not be a forced major version upgrade, unless the bugfix
cannot be easily backported. I for one intend to get the source RPM's for
the fixed packages -- who knows, maybe some of the patches include the
ability to rebuild on later Red Hat Linux versions, helping my upgradability
crusade a little.

As to the 7.2.4 issue, much if not all of our userbase is more than used to
multiple concurrent OS kernel branches. Linux users in particular are very
used to parallel versions -- the 2.0.x and 2.2.x series still get occassional
releases even with 2.4.x out, and the development versions are in parallel
constantly (except during the first few versions of a recent stable) with
stable releases.. FreeBSD has their branches, etc. I think we should
release a 7.2.4 if the bugfixes warrant it. (Not a 7.1.4, though, or 7.0.4,
or 6.5.4, or 6.4.3, or 6.3.3, or....)

And I'm not against progress -- just against forced progress.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen <lamar(dot)owen(at)wgcr(dot)org> writes:
> ... Why? If a user doesn't need the features of 7.x.x, and the codebase is
> working well for him/her, why should said user/DBA feel compelled to go
> through who knows what mechanations to upgrade to the latest version?

Because there are unfixable bugs in the older versions. I see very
little point in issuing "security updates" that fix individual buffer
overruns, when anyone who has the SQL-level access needed to trigger
one of those overruns can equally easily do "select cash_out(2)".
The only fix for that is an upgrade to 7.3.

I don't by any means have a problem with Red Hat issuing maintenance
releases against old versions (nor, as I said, do I have any objection
to a 7.2.4 community release; I just said it wasn't my decision to make).
What I am questioning is the value of fixing some security holes when
there are bigger, unfixable ones right next door. It wastes time that
could be spent on other work, and it may give DBAs a false sense of
security. "Sure I'm safe; I just got the latest security patch from
Red Hat, so my 6.5.3 Postgres must be bulletproof now!"

regards, tom lane

On Saturday 18 January 2003 11:13, Tom Lane wrote:
> Lamar Owen <lamar(dot)owen(at)wgcr(dot)org> writes:
> > ... Why? If a user doesn't need the features of 7.x.x, and the codebase
> > is working well for him/her, why should said user/DBA feel compelled to
> > go through who knows what mechanations to upgrade to the latest version?

> Because there are unfixable bugs in the older versions. I see very
> little point in issuing "security updates" that fix individual buffer
> overruns, when anyone who has the SQL-level access needed to trigger
> one of those overruns can equally easily do "select cash_out(2)".
> The only fix for that is an upgrade to 7.3.

And the cure might be worse than the disease; that is my point.

> It wastes time that
> could be spent on other work, and it may give DBAs a false sense of
> security. "Sure I'm safe; I just got the latest security patch from
> Red Hat, so my 6.5.3 Postgres must be bulletproof now!"

Red Hat issued a very detailed synopsis of what was fixed. Also, one man's
wasted time is another man's time well spent.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

This is an interesting thought. My gut tells me it is a viable
opportunity for the corporate entities that offer support and wish to
have 'VAR' status.

This is just my opinion, but I view the core development group as pure
development, and the various people that resell or distribute PostgreSQL
as a for-profit business as those responsible for maintaining backward
support.

Maybe RedHat or PostgreSQL Inc can do this? It is a really good message,
"The best of open source, with on going support."

And not to re-open a can of worms, but if PostgreSQL could upgrade
without having to do a dump and restore, then this wouldn't really be an
issue.

Justin Clift wrote:

> Hi everyone,
>
> Over the last few days we've had patches submitted for 7.2.3 that
> address a couple of things, both the WAL Recovery Bug that Tom has
> developed a patch for, and a couple of buffer overflows that have been
> widely reported.
>
> Although we haven't wanted to release a 7.2.4, and have instead
> encouraged people to upgrade to 7.3.x, there are places out there
> who's applications aren't compatible with 7.3.x and would also need to
> upgrade them as well.
>
> It might be a really good idea if we re-visit the thought of 7.2.4 and
> have something that people running the 7.2.x series can use safely
> until they are able to move to 7.3.x or above.
>
> What would it take, and apart from patches for the buffer overflows
> and the WAL recovery bug, should anything else be included to ensure
> safety and stability?
>
> :-)
>
> Regards and best wishes,
>
> Justin Clift
>

mlw wrote:
> This is an interesting thought. My gut tells me it is a viable
> opportunity for the corporate entities that offer support and wish to
> have 'VAR' status.
>
> This is just my opinion, but I view the core development group as pure
> development, and the various people that resell or distribute PostgreSQL
> as a for-profit business as those responsible for maintaining backward
> support.
>
> Maybe RedHat or PostgreSQL Inc can do this? It is a really good message,
> "The best of open source, with on going support."

Very interesting thought. It could probably be done. Oh, hang on...
Red Hat is taking that angle for now. :-)

> And not to re-open a can of worms, but if PostgreSQL could upgrade
> without having to do a dump and restore, then this wouldn't really be an
> issue.

That's not really true. Have personally seen applications that places
use and rely on that are not yet compatible with v 7.3.x, because the
vendors of the applications compiled against something that was of
version 7.2.x, and doesn't work with version 7.3.x.

Now, that's not our fault, and not the fault of the places running the
applications, it's just part of how PostgreSQL is applied out in the
real world.

:-)

Regards and best wishes,

Justin Clift

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

Bruce Momjian wrote:
> Tom Lane wrote:
<snip>
>>PS: I'm not taking a position on Justin's suggestion that there should
>>be a 7.2.4. Marc and Bruce would be the ones who have to do the work,
>>so they get to make the decision...
>
> Who, us? Well, there is the confusion factor of releasing a patch to a
> superceeded major version. Wrapping it up and putting it out really
> isn't a big deal. Marc?

Hi Marc,

Would you be ok with us releasing a 7.2.4?

:-)

Regards and best wishes,

Justin Clift

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

On Thu, 2003-01-16 at 22:47, Justin Clift wrote:
> Over the last few days we've had patches submitted for 7.2.3 that
> address a couple of things, both the WAL Recovery Bug that Tom has
> developed a patch for, and a couple of buffer overflows that have been
> widely reported.

The buffer overflows, IMHO, are not sufficient reason to release an
update. As Tom pointed out, there are lots of other, unpatched overflows
in 7.2.3 (and the whole class of vulnerability requires SQL access to
begin with).

As for the "WAL recovery bug", AFAIK no such bug has been reported "in
the last few days". Exactly what issue are you referring to?

Cheers,

Neil

On Sat, 18 Jan 2003, Tom Lane wrote:

> PS: I'm not taking a position on Justin's suggestion that there should
> be a 7.2.4. Marc and Bruce would be the ones who have to do the work,
> so they get to make the decision...

I have no problems creating one ... Bruce?

Neil Conway wrote:
> On Thu, 2003-01-16 at 22:47, Justin Clift wrote:
> > Over the last few days we've had patches submitted for 7.2.3 that
> > address a couple of things, both the WAL Recovery Bug that Tom has
> > developed a patch for, and a couple of buffer overflows that have been
> > widely reported.
>
> The buffer overflows, IMHO, are not sufficient reason to release an
> update. As Tom pointed out, there are lots of other, unpatched overflows
> in 7.2.3 (and the whole class of vulnerability requires SQL access to
> begin with).
>
> As for the "WAL recovery bug", AFAIK no such bug has been reported "in
> the last few days". Exactly what issue are you referring to?

Let's look at the issue here --- I think security fixes are of a
different class from corruption bugs or functionality bugs. For the
latter, fixing those fixes actual problems in the server that actually
improve the capabilities of the database. For security issues, if we
already have ten open doors in a house, does it help to lock two of them
when the other eight are still open? I don't see any improvement in the
functionality of PostgreSQL in such a case, while feature/corruption
fixes _do_ improve the backend code.

I think we have to accept the statement that in 7.2.X malicious SQL
queries can cause database failure, and fixing one or two of the ten
known problems doesn't change that fact.

I don't have a problem with releasing 7.2.4 and including all the fixes,
including security fixes, but I don't see the security fixes _as_ _a_
_reason_ to release a 7.2.4.

So, do we have non-security fixes to warrant a 7.2.X?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Saturday 25 January 2003 20:36, Bruce Momjian wrote:
> improve the capabilities of the database. For security issues, if we
> already have ten open doors in a house, does it help to lock two of them
> when the other eight are still open?

Yes. It depends upon which street the door faces. See the MS SQL Server
Sapphire worm for reference.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen wrote:
> On Saturday 25 January 2003 20:36, Bruce Momjian wrote:
> > improve the capabilities of the database. For security issues, if we
> > already have ten open doors in a house, does it help to lock two of them
> > when the other eight are still open?
>
> Yes. It depends upon which street the door faces. See the MS SQL Server
> Sapphire worm for reference.

Right. All our open doors are on the inside, so we aren't too bad.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Saturday 25 January 2003 21:06, Bruce Momjian wrote:
> Lamar Owen wrote:
> > On Saturday 25 January 2003 20:36, Bruce Momjian wrote:
> > > improve the capabilities of the database. For security issues, if we
> > > already have ten open doors in a house, does it help to lock two of
> > > them when the other eight are still open?

> > Yes. It depends upon which street the door faces. See the MS SQL Server
> > Sapphire worm for reference.

> Right. All our open doors are on the inside, so we aren't too bad.

SQL injection exploits for various frontends are also an issue.

I just have an issue with being able to crash the server with an SQL command.
We'll see how it pans out, I guess.

Red Hat certainly thought it was worth spending some time on; reference their
back porting of the fixes to versions as old as 6.5.3.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen wrote:
> On Saturday 25 January 2003 21:06, Bruce Momjian wrote:
> > Lamar Owen wrote:
> > > On Saturday 25 January 2003 20:36, Bruce Momjian wrote:
> > > > improve the capabilities of the database. For security issues, if we
> > > > already have ten open doors in a house, does it help to lock two of
> > > > them when the other eight are still open?
>
> > > Yes. It depends upon which street the door faces. See the MS SQL Server
> > > Sapphire worm for reference.
>
> > Right. All our open doors are on the inside, so we aren't too bad.
>
> SQL injection exploits for various frontends are also an issue.
>
> I just have an issue with being able to crash the server with an SQL command.
> We'll see how it pans out, I guess.
>
> Red Hat certainly thought it was worth spending some time on; reference their
> back porting of the fixes to versions as old as 6.5.3.

If we can get them all, it is a big win. If we can't, I don't think it
is a win.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> So, do we have non-security fixes to warrant a 7.2.X?

There's the order-of-operations-in-checkpoint problem, and there's
one variant of the "no one parent tuple was found" problem that
should have been patched in 7.2.3, but was overlooked.

Also, the bogus-datetime-table-ordering bugs appear to exist in
7.2 (cf. recent complaint about timezone ART not being recognized).
That ought to be back-patched, if we're going to make a 7.2.4,
though one could certainly say that that doesn't merit a release
by itself.

I think there's enough to warrant a 7.2.4 ...

regards, tom lane

Agreed. How do we get the patches in there, or are they there already?

---------------------------------------------------------------------------

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > So, do we have non-security fixes to warrant a 7.2.X?
>
> There's the order-of-operations-in-checkpoint problem, and there's
> one variant of the "no one parent tuple was found" problem that
> should have been patched in 7.2.3, but was overlooked.
>
> Also, the bogus-datetime-table-ordering bugs appear to exist in
> 7.2 (cf. recent complaint about timezone ART not being recognized).
> That ought to be back-patched, if we're going to make a 7.2.4,
> though one could certainly say that that doesn't merit a release
> by itself.
>
> I think there's enough to warrant a 7.2.4 ...
>
> regards, tom lane
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Agreed. How do we get the patches in there, or are they there already?

We patch ;-). I've been working on it the past few days. Not quite
done, but close.

regards, tom lane

On Sat, Jan 25, 2003 at 09:55:25PM -0500, Bruce Momjian wrote:
> Lamar Owen wrote:
> > We'll see how it pans out, I guess.
> >
> > Red Hat certainly thought it was worth spending some time on; reference their
> > back porting of the fixes to versions as old as 6.5.3.
>
> If we can get them all, it is a big win. If we can't, I don't think it
> is a win.

In the context of backporting, this is true, but in general, if you
don't worry about putting locks on any of the doors, because there are
other ones open, you _never_ get them all.

Ross

Bruce, I've finished digging for stuff that seems to be appropriate to
back-patch for 7.2.4. Do you have time to generate the release notes
and brand the release? Attached are the CVS commit messages for all
the changes in that branch since 7.2.3.

regards, tom lane

2003-01-26 18:16 tgl

* src/backend/commands/user.c (REL7_2_STABLE): Back-patch fixes to
detoast pg_group.grolist.

2003-01-26 18:09 tgl

* src/backend/access/heap/heapam.c (REL7_2_STABLE): Back-patch
fixes to ensure t_ctid always has correct value (prevents some
instances of 'No one parent tuple' VACUUM error, and perhaps worse
things).

2003-01-26 17:33 tgl

* src/: backend/utils/adt/datetime.c,
test/regress/expected/timestamp.out,
test/regress/expected/timestamptz.out (REL7_2_STABLE): Back-patch
fix for alphabetization mistakes in datetime token tables.

2003-01-21 14:51 tgl

* src/backend/access/transam/xlog.c (REL7_2_STABLE): Back-patch fix
to ensure pg_clog updates are not only written but sync'ed before
we consider the checkpoint to be done.

2003-01-21 14:41 tgl

* src/backend/utils/adt/geo_ops.c (REL7_2_STABLE): Back-patch fixes
for integer overflows in circle_poly(), path_encode(), and
path_add() --- from Neil Conway. Also, repair recently-detected
errors in lseg_eq(), lseg_ne(), lseg_center().

2003-01-21 14:38 tgl

* src/backend/commands/vacuum.c (REL7_2_STABLE): Back-patch fix for
VACUUM being confused by SELECT FOR UPDATE of tuple that was
previously outdated by a transaction that later aborted. Also,
prevent VACUUM from being called inside function.

--
Dave Cramer <dave(at)fastcrypt(dot)com>
Cramer Consulting

This is useful for some O/R tools. The JDBC spec has a getTableName method for each column in a result set.

One issue which will come up is what to do with aggregate, and computed values. For now, we could return null

So for a "select a, b, a+b as sum from c" returns c.a, c.b, ?table?.sum

Dave

Dave Cramer <dave(at)fastcrypt(dot)com> writes:
> So for a "select a, b, a+b as sum from c" returns c.a, c.b, ?table?.sum

This might be something to consider as part of the planned protocol
overhaul. We cannot simply change the returned column names --- at
least not without breaking a lot of application code. But if we
return table name (and schema name too!) as separate fields of the
'T' message, and make them accessible through new PQfoo accessor
functions, then no existing applications would break.

But there are more than a few definitional issues to be settled before
you'll convince me this idea is fully baked. Some things that come to
mind immediately:

What happens with views? Given
create view v as select col as vcol from tab;
select vcol from v;
are you expecting to get back "v.vcol"? Or "tab.col"?

What happens with FROM-clause aliases? Supposing tab really has a
column "col", what do you expect to see from
select * from tab AS a(t1), tab AS b(t2) WHERE ...
You could make a case for either "tab.col, tab.col" or "a.t1, b.t2"
(in the latter case, you can't realistically return a schema name).
But you will probably break existing code if you do the former, since
currently the output columns are labeled t1, t2.

What happens with join aliases (similar issues to above)?

Do you think
select col as foo from tab
should return "tab.foo", or just "foo"? I'd lean to the latter;
"tab.foo" seems awfully misleading. Or maybe you're wanting it
to ignore the AS and return "tab.col"? Don't think that will fly.

regards, tom lane

> I think we have to accept the statement that in 7.2.X malicious SQL
> queries can cause database failure, and fixing one or two of the ten
> known problems doesn't change that fact.
>
> I don't have a problem with releasing 7.2.4 and including all the fixes,
> including security fixes, but I don't see the security fixes _as_ _a_
> _reason_ to release a 7.2.4.
>
> So, do we have non-security fixes to warrant a 7.2.X?

Gavin Sherry and I have just spent a week at the Linux.conf.au. The
feedback we got from users was basically this:

1. We don't allow untrusted users unlimited SQL access
2. Upgrading PostgreSQL sucks
3. We want important corruption fixes
4. So, keep supporting older versions (7.2.x at least)

So, basically I think it is a VERY good idea for us to keep releasing 7.2.x
versions for a long time.

BTW, I'll be posting a linux.conf.au postgres report soonish...

Chris

When talking about expressions,views, or any other construct that could
combine values from multiple tables I think it is reasonable to provide
null as the table name. Any one or any process requesting the table
name has to understand that not all SQL parameters have a base table
name. However, in the case where a single table is involved, table and
schema names should be available.

Reggie

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org [mailto:pgsql-hackers-
> owner(at)postgresql(dot)org] On Behalf Of Tom Lane
> Sent: Sunday, January 26, 2003 7:39 PM
> To: Dave Cramer
> Cc: PostgreSQL Hackers Mailing List
> Subject: Re: [HACKERS] Request for qualified column names
>
> Dave Cramer <dave(at)fastcrypt(dot)com> writes:
> > So for a "select a, b, a+b as sum from c" returns c.a, c.b,
?table?.sum
>
> This might be something to consider as part of the planned protocol
> overhaul. We cannot simply change the returned column names --- at
> least not without breaking a lot of application code. But if we
> return table name (and schema name too!) as separate fields of the
> 'T' message, and make them accessible through new PQfoo accessor
> functions, then no existing applications would break.
>
> But there are more than a few definitional issues to be settled before
> you'll convince me this idea is fully baked. Some things that come to
> mind immediately:
>
> What happens with views? Given
> create view v as select col as vcol from tab;
> select vcol from v;
> are you expecting to get back "v.vcol"? Or "tab.col"?
>
> What happens with FROM-clause aliases? Supposing tab really has a
> column "col", what do you expect to see from
> select * from tab AS a(t1), tab AS b(t2) WHERE ...
> You could make a case for either "tab.col, tab.col" or "a.t1, b.t2"
> (in the latter case, you can't realistically return a schema name).
> But you will probably break existing code if you do the former, since
> currently the output columns are labeled t1, t2.
>
> What happens with join aliases (similar issues to above)?
>
> Do you think
> select col as foo from tab
> should return "tab.foo", or just "foo"? I'd lean to the latter;
> "tab.foo" seems awfully misleading. Or maybe you're wanting it
> to ignore the AS and return "tab.col"? Don't think that will fly.
>
> regards, tom lane
>
> ---------------------------(end of
broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster

"Reggie Burnett" <rykr(at)bellsouth(dot)net> writes:
> When talking about expressions,views, or any other construct that could
> combine values from multiple tables I think it is reasonable to provide
> null as the table name. Any one or any process requesting the table
> name has to understand that not all SQL parameters have a base table
> name. However, in the case where a single table is involved, table and
> schema names should be available.

That seems quite pointless. You hardly need the backend's help to
determine which column belongs to which table in a single-table query.
AFAICS this facility is only of interest if it does something useful
in not-so-trivial cases.

regards, tom lane

Well, certainly the driver could parse the sql and extract what it
thinks is the table name. It just seems quite foreign to me to have a
database engine go through the motions of determining column location
and have ready access to all the metadata for all the columns in a
resultset and then intentionally leave all that out of the FE/BE. Now,
for us driver writers, if I have a select statement that has 20 columns
I will need to extract the tablename myself (and hope I got it right)
and then execute 20 separate queries to the database in order to
implement any type of schema generation. I guess I don't understand
this when just a few extra bytes in the RowDescriptor message would have
fixed all this.

Reggie

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org [mailto:pgsql-hackers-
> owner(at)postgresql(dot)org] On Behalf Of Tom Lane
> Sent: Monday, January 27, 2003 9:21 AM
> To: Reggie Burnett
> Cc: 'Dave Cramer'; 'PostgreSQL Hackers Mailing List'
> Subject: Re: [HACKERS] Request for qualified column names
>
> "Reggie Burnett" <rykr(at)bellsouth(dot)net> writes:
> > When talking about expressions,views, or any other construct that
could
> > combine values from multiple tables I think it is reasonable to
provide
> > null as the table name. Any one or any process requesting the table
> > name has to understand that not all SQL parameters have a base table
> > name. However, in the case where a single table is involved, table
and
> > schema names should be available.
>
> That seems quite pointless. You hardly need the backend's help to
> determine which column belongs to which table in a single-table query.
> AFAICS this facility is only of interest if it does something useful
> in not-so-trivial cases.
>
> regards, tom lane
>
> ---------------------------(end of
broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org

My idea on this after chat with Dave was to add a GUC option that puts
the schema.table.column name as the default column label, rather than
just the column name. (That's so easy, I think even I could do it.) If
they over-ride it with AS, or if it is an aggregate or FROM subquery, we
just return the default label as we do now --- we could return no label
for those cases, but that seems too drastic. I am not overly excited
about doing this at the protocol level unless there is major need for it.

---------------------------------------------------------------------------

Tom Lane wrote:
> "Reggie Burnett" <rykr(at)bellsouth(dot)net> writes:
> > When talking about expressions,views, or any other construct that could
> > combine values from multiple tables I think it is reasonable to provide
> > null as the table name. Any one or any process requesting the table
> > name has to understand that not all SQL parameters have a base table
> > name. However, in the case where a single table is involved, table and
> > schema names should be available.
>
> That seems quite pointless. You hardly need the backend's help to
> determine which column belongs to which table in a single-table query.
> AFAICS this facility is only of interest if it does something useful
> in not-so-trivial cases.
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

--On Monday, January 27, 2003 15:49:06 -0500 Bruce Momjian
<pgman(at)candle(dot)pha(dot)pa(dot)us> wrote:

>
> My idea on this after chat with Dave was to add a GUC option that puts
> the schema.table.column name as the default column label, rather than
> just the column name. (That's so easy, I think even I could do it.) If
> they over-ride it with AS, or if it is an aggregate or FROM subquery, we
> just return the default label as we do now --- we could return no label
> for those cases, but that seems too drastic. I am not overly excited
> about doing this at the protocol level unless there is major need for it.
DONT DEFAULT TO THE NEW ONE WITHOUT NOTICE!

You will ***BREAK*** people.

LER

>
> -------------------------------------------------------------------------
> --
>
> Tom Lane wrote:
>> "Reggie Burnett" <rykr(at)bellsouth(dot)net> writes:
>> > When talking about expressions,views, or any other construct that could
>> > combine values from multiple tables I think it is reasonable to provide
>> > null as the table name. Any one or any process requesting the table
>> > name has to understand that not all SQL parameters have a base table
>> > name. However, in the case where a single table is involved, table and
>> > schema names should be available.
>>
>> That seems quite pointless. You hardly need the backend's help to
>> determine which column belongs to which table in a single-table query.
>> AFAICS this facility is only of interest if it does something useful
>> in not-so-trivial cases.
>>
>> regards, tom lane
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 6: Have you searched our list archives?
>>
>> http://archives.postgresql.org
>>
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania
> 19073
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
>

--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler(at)lerctr(dot)org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> My idea on this after chat with Dave was to add a GUC option that puts
> the schema.table.column name as the default column label, rather than
> just the column name.

And will you quotify things so that names containing dots, spaces, etc
are unambiguous?

I think the above is a very poor substitute for doing it properly,
namely returning the values in separate fields. We should not allow
ourselves to get lured into a dead end just because we can do it without
obviously breaking the protocol. (I would argue that this breaks the
protocol anyway, though.)

> I am not overly excited
> about doing this at the protocol level unless there is major need for it.

I'm not excited about doing it at all, unless we do it right. We can
already have half-baked solutions on the client side ;-)

regards, tom lane

Larry Rosenman wrote:
>
>
> --On Monday, January 27, 2003 15:49:06 -0500 Bruce Momjian
> <pgman(at)candle(dot)pha(dot)pa(dot)us> wrote:
>
> >
> > My idea on this after chat with Dave was to add a GUC option that puts
> > the schema.table.column name as the default column label, rather than
> > just the column name. (That's so easy, I think even I could do it.) If
> > they over-ride it with AS, or if it is an aggregate or FROM subquery, we
> > just return the default label as we do now --- we could return no label
> > for those cases, but that seems too drastic. I am not overly excited
> > about doing this at the protocol level unless there is major need for it.
> DONT DEFAULT TO THE NEW ONE WITHOUT NOTICE!
>
> You will ***BREAK*** people.

Of course we are not going to default this to ON.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > My idea on this after chat with Dave was to add a GUC option that puts
> > the schema.table.column name as the default column label, rather than
> > just the column name.
>
> And will you quotify things so that names containing dots, spaces, etc
> are unambiguous?
>
> I think the above is a very poor substitute for doing it properly,
> namely returning the values in separate fields. We should not allow
> ourselves to get lured into a dead end just because we can do it without
> obviously breaking the protocol. (I would argue that this breaks the
> protocol anyway, though.)

I don't see how it is worth modifying the client or protocol unless we
have more demand for it. I would quote the labels, yes.

> > I am not overly excited
> > about doing this at the protocol level unless there is major need for it.
>
> I'm not excited about doing it at all, unless we do it right. We can
> already have half-baked solutions on the client side ;-)

It is easy on the server, quite hard on the client.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Mon, 2003-01-27 at 15:50, Larry Rosenman wrote:
> --On Monday, January 27, 2003 15:49:06 -0500 Bruce Momjian
> <pgman(at)candle(dot)pha(dot)pa(dot)us> wrote:
>
> >
> > My idea on this after chat with Dave was to add a GUC option that puts
> > the schema.table.column name as the default column label, rather than
> > just the column name. (That's so easy, I think even I could do it.) If
> > they over-ride it with AS, or if it is an aggregate or FROM subquery, we
> > just return the default label as we do now --- we could return no label
> > for those cases, but that seems too drastic. I am not overly excited
> > about doing this at the protocol level unless there is major need for it.
> DONT DEFAULT TO THE NEW ONE WITHOUT NOTICE!
>
> You will ***BREAK*** people.

Agreed. This is the way we probably want to go -- but we'll need a guc
for a release or 2 -- One release with default as current, one with
default as new way, 7.6 can remove Guc.

--
Rod Taylor <rbt(at)rbt(dot)ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Bruce Momjian writes:

> My idea on this after chat with Dave was to add a GUC option that puts
> the schema.table.column name as the default column label, rather than
> just the column name.

Can someone explain why this is needed at all? There is a reason why the
SQL standard does not provide for this information: it's not well defined.
Are you trying to make up a poor substitute for updatable views?

--
Peter Eisentraut peter_e(at)gmx(dot)net

Dave Cramer says it is needed for the jdbc spec, somehow. It seems kind
of odd so I don't want to make too complex an implementation.

---------------------------------------------------------------------------

Peter Eisentraut wrote:
> Bruce Momjian writes:
>
> > My idea on this after chat with Dave was to add a GUC option that puts
> > the schema.table.column name as the default column label, rather than
> > just the column name.
>
> Can someone explain why this is needed at all? There is a reason why the
> SQL standard does not provide for this information: it's not well defined.
> Are you trying to make up a poor substitute for updatable views?
>
> --
> Peter Eisentraut peter_e(at)gmx(dot)net
>
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Dave Cramer says it is needed for the jdbc spec, somehow.

Does the JDBC spec really require the database to provide functionality
that's not in the SQL spec? I kinda doubt that.

regards, tom lane

What is needed, at least from my perspective, is a way to determine
proper meta data for a given column. Is it updatable? Is it nullable?
Is it part of a primary key or index? Without either the base table
name or attrelid,indrelid of the table, I can't get this info.

Reggie

> -----Original Message-----
> From: Bruce Momjian [mailto:pgman(at)candle(dot)pha(dot)pa(dot)us]
> Sent: Tuesday, January 28, 2003 4:01 PM
> To: Peter Eisentraut
> Cc: Tom Lane; Reggie Burnett; 'Dave Cramer'; 'PostgreSQL Hackers
Mailing
> List'
> Subject: Re: [HACKERS] Request for qualified column names
>
>
> Dave Cramer says it is needed for the jdbc spec, somehow. It seems
kind
> of odd so I don't want to make too complex an implementation.
>
>
------------------------------------------------------------------------
--
> -
>
> Peter Eisentraut wrote:
> > Bruce Momjian writes:
> >
> > > My idea on this after chat with Dave was to add a GUC option that
puts
> > > the schema.table.column name as the default column label, rather
than
> > > just the column name.
> >
> > Can someone explain why this is needed at all? There is a reason
why
> the
> > SQL standard does not provide for this information: it's not well
> defined.
> > Are you trying to make up a poor substitute for updatable views?
> >
> > --
> > Peter Eisentraut peter_e(at)gmx(dot)net
> >
> >
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania
> 19073

Could someone point me to this standard? Is that the standard for SQL
syntax? I wasn't aware there was a standard for RDBMS functionality. I
always assumed the features provided by the RDBMS were up to the
implementers.

Reggie

> -----Original Message-----
> From: Peter Eisentraut [mailto:peter_e(at)gmx(dot)net]
> Sent: Tuesday, January 28, 2003 3:59 PM
> To: Bruce Momjian
> Cc: Tom Lane; Reggie Burnett; 'Dave Cramer'; 'PostgreSQL Hackers
Mailing
> List'
> Subject: Re: [HACKERS] Request for qualified column names
>
> Bruce Momjian writes:
>
> > My idea on this after chat with Dave was to add a GUC option that
puts
> > the schema.table.column name as the default column label, rather
than
> > just the column name.
>
> Can someone explain why this is needed at all? There is a reason why
the
> SQL standard does not provide for this information: it's not well
defined.
> Are you trying to make up a poor substitute for updatable views?
>
> --
> Peter Eisentraut peter_e(at)gmx(dot)net

The method in question is

ResultSetMetaDate.getTableName(int column)
and while were at it

ResultSetMetaData.getSchemaName(int column)

and FWIW, the return value if not applicable is ""

Dave
On Tue, 2003-01-28 at 17:21, Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Dave Cramer says it is needed for the jdbc spec, somehow.
>
> Does the JDBC spec really require the database to provide functionality
> that's not in the SQL spec? I kinda doubt that.
>
> regards, tom lane
--
Dave Cramer <dave(at)fastcrypt(dot)com>
Cramer Consulting

On Mon, 2003-01-27 at 10:44, Reggie Burnett wrote:
> Well, certainly the driver could parse the sql and extract what it
> thinks is the table name. It just seems quite foreign to me to have a
> database engine go through the motions of determining column location
> and have ready access to all the metadata for all the columns in a
> resultset and then intentionally leave all that out of the FE/BE.

I think the issue is that no one has yet proposed a consistent set of
behaviour for this feature, particularly in the cases that Tom raised.
If you would like this feature, I'd suggest that you outline some
behaviour that everyone can agree upon.

Griping about "intentionally left out" features when the feature itself
is not even well defined doesn't strike me as very productive.

Cheers,

Neil
--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC

"Ross J. Reedstrom" <reedstrm(at)rice(dot)edu> writes:
> On Sat, Jan 25, 2003 at 09:55:25PM -0500, Bruce Momjian wrote:
>> If we can get them all, it is a big win. If we can't, I don't think it
>> is a win.

> In the context of backporting, this is true, but in general, if you
> don't worry about putting locks on any of the doors, because there are
> other ones open, you _never_ get them all.

We certainly are trying to get them all going forward. The issue here
is what is reasonable to back-patch into 7.2 (or 7.3), given the ground
rules that we can no longer force an initdb for users of those releases.
Those ground rules mean that some bugs are unfixable in those releases.

How hard should we try to back-patch fixes for fixable bugs of severity
comparable to the unfixable bugs? Before you answer, consider that any
time spent doing so takes away from current/future development work;
"fix it without regard to cost" is not really a defensible stance.

regards, tom lane

I'm certainly not trying to be difficult, I just don't know a lot about
the internals of PostgreSQL. I'm developing some interfaces to various
databases and certainly wanted to include PostgreSQL.

From my less-than-qualified viewpoint, I would have thought including
the base table name and bit pattern indicating certain features
(nullability, primary index, etc) for each column in the RowDescriptor
message would have been the best. Since my driver will need to support
current and previous versions of PostgreSQL, my plan is to write some
code to parse a SQL statement and extract the table names. (ugh!)

One approach might be to add the tables's oid to the RowDescriptor
message. Would not be perfect since I still would have many roundtrips
to the database to get metadata, but since I don't need metadata in
every case I can leave that step out until someone requests it.

Reggie

> -----Original Message-----
> From: Neil Conway [mailto:neilc(at)samurai(dot)com]
> Sent: Tuesday, January 28, 2003 11:47 PM
> To: Reggie Burnett
> Cc: 'Tom Lane'; 'Dave Cramer'; 'PostgreSQL Hackers Mailing List'
> Subject: Re: [HACKERS] Request for qualified column names
>
> On Mon, 2003-01-27 at 10:44, Reggie Burnett wrote:
> > Well, certainly the driver could parse the sql and extract what it
> > thinks is the table name. It just seems quite foreign to me to have
a
> > database engine go through the motions of determining column
location
> > and have ready access to all the metadata for all the columns in a
> > resultset and then intentionally leave all that out of the FE/BE.
>
> I think the issue is that no one has yet proposed a consistent set of
> behaviour for this feature, particularly in the cases that Tom raised.
> If you would like this feature, I'd suggest that you outline some
> behaviour that everyone can agree upon.
>
> Griping about "intentionally left out" features when the feature
itself
> is not even well defined doesn't strike me as very productive.
>
> Cheers,
>
> Neil
> --
> Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC
>
>

Dave Cramer writes:

> The method in question is
> ResultSetMetaDate.getTableName(int column)
> and while were at it
> ResultSetMetaData.getSchemaName(int column)
> and FWIW, the return value if not applicable is ""

Not applicable sounds fine to me. It's like taking a file descriptor and
asking what file it belongs to. That information simply doesn't exist,
and if you design an application around it you lose.

--
Peter Eisentraut peter_e(at)gmx(dot)net

Peter Eisentraut wrote:
> Dave Cramer writes:
>
> > The method in question is
> > ResultSetMetaDate.getTableName(int column)
> > and while were at it
> > ResultSetMetaData.getSchemaName(int column)
> > and FWIW, the return value if not applicable is ""
>
> Not applicable sounds fine to me. It's like taking a file descriptor and
> asking what file it belongs to. That information simply doesn't exist,
> and if you design an application around it you lose.

Yes, but in cases we can supply the info with the proper GUC variable
enabled, why not do it? I realize most people don't want it, but if
jdbc does, and it is something folks would use, maybe we should enable
it for the easy cases.

However, the number of cases where we would not be able to easily report
it may make the feature useless.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073