Re: Open 7.3 items

Here are the open items for 7.3. We have one more month to address them
before beta.

---------------------------------------------------------------------------

P O S T G R E S Q L

7 . 3 O P E N I T E M S

Current at ftp://candle.pha.pa.us/pub/postgresql/open_items.

Source Code Changes
-------------------
Socket permissions - only install user can access db by default
unix_socket_permissions in postgresql.conf
NAMEDATALEN - disk/performance penalty for increase, 64, 128?
FUNC_MAX_ARGS - disk/performance penalty for increase, 24, 32?
Point-in-time recovery - ready for 7.3?
Allow easy display of usernames in a group (pg_hba.conf uses groups now)
Reindex/btree shrinkage - does reindex need work, can btree be shrunk?
DROP COLUMN - ready?
CLUSTER - ready?
display locks - ready?
Win32 - timefame?
Prepared statements - ready?
Schema handling - ready? interfaces? client apps?
Dependency - pg_dump auto-create dependencies for 7.2.X data?
glibc and mktime() - fix?
Functions Returning Sets - done?
ecpg and bison issues - solved?

Documentation Changes
---------------------

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

> Schema handling - ready? interfaces? client apps?

With schemas, how about settings for automatically creating a schema for a
user when you create the user, etc.

Chris

On Tuesday 30 July 2002 11:50 pm, Bruce Momjian wrote:
> Here are the open items for 7.3. We have one more month to address them
> before beta.

> Source Code Changes
> -------------------

Bruce, is the config file location stuff not being addressed? I remember Mark
(mlw) had worked up the patch for a '-C' switch, there was discussion, etc.
Did it not make the todo?

If Peter or someone else doesn't beat me to it I might try my hand at that
one, as I would dearly love to be able to decouple the config files from
PGDATA. It has been discussed; consensus was not reached as I recall.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Lamar Owen wrote:
> On Tuesday 30 July 2002 11:50 pm, Bruce Momjian wrote:
> > Here are the open items for 7.3. We have one more month to address them
> > before beta.
>
> > Source Code Changes
> > -------------------
>
> Bruce, is the config file location stuff not being addressed? I remember Mark
> (mlw) had worked up the patch for a '-C' switch, there was discussion, etc.
> Did it not make the todo?
>
> If Peter or someone else doesn't beat me to it I might try my hand at that
> one, as I would dearly love to be able to decouple the config files from
> PGDATA. It has been discussed; consensus was not reached as I recall.

The issue was never resolved, so it did not make any lists.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Lamar Owen <lamar(dot)owen(at)wgcr(dot)org> writes:
> Bruce, is the config file location stuff not being addressed? ...
> If Peter or someone else doesn't beat me to it I might try my hand at that
> one, as I would dearly love to be able to decouple the config files from
> PGDATA. It has been discussed; consensus was not reached as I recall.

I believe that last part was the sticking point. If you can find some
consensus on how it ought to work, then go for it. My own opinion is
that there is nothing broken there; certainly nothing so broken that
we need to force a change under schedule pressure.

regards, tom lane

Tom Lane wrote:
> Lamar Owen <lamar(dot)owen(at)wgcr(dot)org> writes:
> > Bruce, is the config file location stuff not being addressed? ...
> > If Peter or someone else doesn't beat me to it I might try my hand at that
> > one, as I would dearly love to be able to decouple the config files from
> > PGDATA. It has been discussed; consensus was not reached as I recall.
>
> I believe that last part was the sticking point. If you can find some
> consensus on how it ought to work, then go for it. My own opinion is
> that there is nothing broken there; certainly nothing so broken that
> we need to force a change under schedule pressure.

I don't feel we are under pressure. We have time to discuss and address
it.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
>> ... My own opinion is
>> that there is nothing broken there; certainly nothing so broken that
>> we need to force a change under schedule pressure.

> I don't feel we are under pressure. We have time to discuss and address
> it.

Well, it's all a matter of how you look at it. Isn't the point of your
post that began this thread to start giving people a sense of time
pressure?

I agree that if we could quickly come to a resolution about how this
ought to work, there's plenty of time to go off and implement it. But
(1) we failed to come to a consensus before, so I'm not optimistic
than one will suddenly emerge now; (2) we've got a ton of other issues
that we *need* to deal with before beta. This one does not strike me
as a must-fix, and so I'm loathe to spend much development time on it
when there are so many open issues.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> ... My own opinion is
> >> that there is nothing broken there; certainly nothing so broken that
> >> we need to force a change under schedule pressure.
>
> > I don't feel we are under pressure. We have time to discuss and address
> > it.
>
> Well, it's all a matter of how you look at it. Isn't the point of your
> post that began this thread to start giving people a sense of time
> pressure?
>
> I agree that if we could quickly come to a resolution about how this
> ought to work, there's plenty of time to go off and implement it. But
> (1) we failed to come to a consensus before, so I'm not optimistic
> than one will suddenly emerge now; (2) we've got a ton of other issues
> that we *need* to deal with before beta. This one does not strike me
> as a must-fix, and so I'm loathe to spend much development time on it
> when there are so many open issues.

Well, it is up to the individuals involved. If someone wants to deal
with it, and gets a consensus, and comes up with a patch, let them.

The list is for time pressure on those items. It does not affect new
items.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Here are the open items for 7.3.

Some comments ...

> Socket permissions - only install user can access db by default

I do not agree with this goal.

> NAMEDATALEN - disk/performance penalty for increase, 64, 128?
> FUNC_MAX_ARGS - disk/performance penalty for increase, 24, 32?

At the moment I don't see a lot of solid evidence that increasing
NAMEDATALEN has any performance penalty. Someone reported about
a 10% slowdown on pgbench with NAMEDATALEN=128 ... but Neil Conway
tried to reproduce the result, and got about a 10% *speedup*.
Personally I think 10% is well within the noise spectrum for
pgbench, and so it's difficult to claim that we have established
any performance difference at all. I have not tried to measure
FUNC_MAX_ARGS differences.

> Point-in-time recovery - ready for 7.3?

At the moment, it doesn't exist at all. If patches appear, we can
review 'em, but right now there is nothing to debate.

> DROP COLUMN - ready?

I'm on it.

> Win32 - timefame?

I've seen nothing to make me think this will be ready for 7.3.

> Prepared statements - ready?

I think we're close there; the patch seems okay, we're just debating
minor syntax issues.

> Schema handling - ready? interfaces? client apps?

The backend will be ready (it's not quite yet). pg_dump is ready.
psql is very definitely not ready, nor is pgaccess. I don't know the
status for JDBC or ODBC; any comments? The other interface libraries
probably don't care.

> Dependency - pg_dump auto-create dependencies for 7.2.X data?

Huh?

> glibc and mktime() - fix?

We need a fix for this. Dunno what to do about it.

> ecpg and bison issues - solved?

Not yet :-(. Anyone have a line into the bison project?

Other things on my radar screen:

* I have about zero confidence in the recent tuple-header-size-reduction
patches.

* pg_conversion stuff --- do we understand this thing's behavior under
failure conditions? Does it work properly with namespaces and
dependencies?

* pg_dumpall probably ought to dump database privilege settings, also
per-user and per-database GUC settings.

* BeOS and QNX4 ports are busted.

* The whole area of which implicit coercions should be allowed is a
serious problem that we have not spent enough time on. There are
a number of cases in which CVS tip is clearly broken compared to
prior releases.

* Bear Giles' SSL patches seem to be causing unhappiness in some
quarters.

* libpqxx is not integrated into build process nor docs. It should
be integrated or reversed out before beta.

regards, tom lane

> * pg_conversion stuff --- do we understand this thing's behavior under
> failure conditions?

As far as I know, automatic encoding conversion behaves well under
failure conditions.

> Does it work properly with namespaces and
> dependencies?

Yes.
--
Tatsuo Ishii

add in 'fix pg_hba.conf / password issues' to that too :)

On Tue, 30 Jul 2002, Bruce Momjian wrote:

>
> Here are the open items for 7.3. We have one more month to address them
> before beta.
>
> ---------------------------------------------------------------------------
>
> P O S T G R E S Q L
>
> 7 . 3 O P E N I T E M S
>
>
> Current at ftp://candle.pha.pa.us/pub/postgresql/open_items.
>
> Source Code Changes
> -------------------
> Socket permissions - only install user can access db by default
> unix_socket_permissions in postgresql.conf
> NAMEDATALEN - disk/performance penalty for increase, 64, 128?
> FUNC_MAX_ARGS - disk/performance penalty for increase, 24, 32?
> Point-in-time recovery - ready for 7.3?
> Allow easy display of usernames in a group (pg_hba.conf uses groups now)
> Reindex/btree shrinkage - does reindex need work, can btree be shrunk?
> DROP COLUMN - ready?
> CLUSTER - ready?
> display locks - ready?
> Win32 - timefame?
> Prepared statements - ready?
> Schema handling - ready? interfaces? client apps?
> Dependency - pg_dump auto-create dependencies for 7.2.X data?
> glibc and mktime() - fix?
> Functions Returning Sets - done?
> ecpg and bison issues - solved?
>
>
> Documentation Changes
> ---------------------
>
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
> + If your life is a hard drive, | 830 Blythe Avenue
> + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> message can get through to the mailing list cleanly
>

On Wed, 31 Jul 2002, Tom Lane wrote:

> * libpqxx is not integrated into build process nor docs. It should
> be integrated or reversed out before beta.

I've requestsed that Jeorgen(sp?) move this over to GBorg ... its
something that can, and should be, built seperately from the base
distribution, along with at least a dozen other things we have bloating
the distribution now :( but at least that one hasn't been integrated yet
...

We got into 'creeping featurisms' in the beginning because we had no other
really central location to put stuff ... we do now, and with better
mechanisms in place for dealing with 'multiple maintainers' ... its
about time we start to trim the fat, before we can't fit through the
proverbial door anymore ...

Peronally, I find it quite annoying to have to download the complete
server distribution just to get libpq installed so that I can install
mod_php4 ... and with all the talk about 'why mysql vs pgsql' that has
been going on lately, its time to start look at how to make it easier to
'add a new interface' without having to download the whole distribution
'yet again' ...

> > Dependency - pg_dump auto-create dependencies for 7.2.X data?
>
> Huh?

Taking a bunch of CREATE CONSTRAINT TRIGGERS and turning them into the
proper pg_constraint entries...

Chris

...
> I agree that if we could quickly come to a resolution about how this
> ought to work, there's plenty of time to go off and implement it. But
> (1) we failed to come to a consensus before, so I'm not optimistic
> than one will suddenly emerge now; (2) we've got a ton of other issues
> that we *need* to deal with before beta. This one does not strike me
> as a must-fix, and so I'm loathe to spend much development time on it
> when there are so many open issues.

afaict someone else volunteered to do the work. There is no lack of
consensus that this is a useful feature, at least among those who take
responsibility to package PostgreSQL for particular platforms. How about
letting them specify the requirements and if an acceptable solution
emerges soon, we'll have it for 7.3...

- Thomas

>> Schema handling - ready? interfaces? client apps?
> status for JDBC or ODBC; any comments? The other interface libraries
> probably don't care.

What about DBD::Pg?

--
Kaare Rasmussen --Linux, spil,-- Tlf: 3816 2582
Kaki Data tshirts, merchandize Fax: 3816 2501
Howitzvej 75 Åben 12.00-18.00 Web: www.suse.dk
2000 Frederiksberg Lørdag 12.00-16.00 Email:kar(at)kakidata(dot)dk

Le Mercredi 31 Juillet 2002 05:50, Bruce Momjian a écrit :
> Source Code Changes
What about CREATE OR REPLACE VIEW which would be great for pgAdmin2. Thanks to
all of you./Jean-Michel POURE

I too do not like alot of bloat in the distribution, but I also agree
with what Andrew is saying.

Currently, at the FTP site, you can download the whole tar file, or in 4
separate tarballs. How hard would it be to create a separate tarball for
client related packages? I am not sure if this would be a *great*
solution, but it might be a good compromise.

--brett

On Wed, 2002-07-31 at 10:22, Andrew Sullivan wrote:
> On Wed, Jul 31, 2002 at 02:08:33PM -0300, Marc G. Fournier wrote:
> > On Wed, 31 Jul 2002, Tom Lane wrote:
>
> > > One reason for wanting to integrate libpqxx is that I don't think we'll
> > > find out anything about its portability until we get a lot of people
> > > trying to build it. If it's a separate distro that won't happen quickly.
> >
> > Who cares? Those that need a C++ interface will know where to find it,
> > and will report bugs that they have ... why should it be tested on every
> > platform when we *might* only have those on the Linux platform using it?
>
> This seems a bad argument. You can't say "we support interface xyz"
> and never test it on anything except i80x86 Linux. Somebady comes
> along and tries to make it go on Solaris, and it doesn't work: poof,
> the cross-platform reputation that you and other have worked hard to
> burnish goes away. Never mind that it's only a client library.
>
> Besides, more generally, Postgres already has a reputation as being
> difficult to install. The proposal to separate out all the
> "non-basics" (I'm not even sure how one would draw that line: maybe a
> server-only package and a client-library package run through GBorg?)
> would mean that anyone wanting to do something moderately complicated
> would have a yet higher hurdle. Isn't that a problem?
>
> A
>
> --
> ----
> Andrew Sullivan 87 Mowat Avenue
> Liberty RMS Toronto, Ontario Canada
> <andrew(at)libertyrms(dot)info> M6K 3E3
> +1 416 646 3304 x110
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
--
Brett Schwarz
brett_schwarz AT yahoo.com

On Wed, Jul 31, 2002 at 02:11:06AM -0300, Marc G. Fournier wrote:
> On Wed, 31 Jul 2002, Tom Lane wrote:
> > * libpqxx is not integrated into build process nor docs. It should
> > be integrated or reversed out before beta.
>
> I've requestsed that Jeorgen(sp?) move this over to GBorg ... its
> something that can, and should be, built seperately from the base
> distribution, along with at least a dozen other things we have bloating
> the distribution now :( but at least that one hasn't been integrated yet
> ...

Mentioning that on -hackers would have been nice -- I've spent a while
this week hacking autoconf / Makefiles to integrate libpqxx...

The problem I have with removing libpqxx is that libpq++ is a far
inferior C++ interface. If we leave libpq++ as the only C++ interface
distributed with PostgreSQL, there will be a tendancy for people
using PostgreSQL & C++ to use the C++ support included with the
distribution. Similarly, the Perl interface included with
PostgreSQL is widely regarded as inferior to DBD::Pg.

If we're going to start removing interfaces, I'd vote for the removal of
perl5 & libpq++ as well as libpqxx. We can add a section to the
documentation listing the available language interfaces (for languages
like Python, where several interfaces exist, this would probably be a
good idea anyway).

Cheers,

Neil

--
Neil Conway <neilconway(at)rogers(dot)com>
PGP Key ID: DB3C29FC

nconway(at)klamath(dot)dyndns(dot)org (Neil Conway) writes:
> Mentioning that on -hackers would have been nice -- I've spent a while
> this week hacking autoconf / Makefiles to integrate libpqxx...

Marc's opinion is not the same thing as a done deal ;-) --- we still
have to discuss this, and if someone's already doing the integration
work I think that's an important factor.

> If we're going to start removing interfaces, I'd vote for the removal of
> perl5 & libpq++ as well as libpqxx.

Agreed on that point. We shouldn't be promoting old, crufty interface
libraries when there are better ones available.

I would personally prefer to see libpqxx integrated now, and then we
could plan to remove libpq++ in a release or two (after giving people
a reasonable opportunity to switch over). If anyone still cares about
libpq++ at that point, it could be given a home on gborg.

One reason for wanting to integrate libpqxx is that I don't think we'll
find out anything about its portability until we get a lot of people
trying to build it. If it's a separate distro that won't happen quickly.

regards, tom lane

On Tue, Jul 30, 2002 at 11:50:38PM -0400, Bruce Momjian wrote:
> NAMEDATALEN - disk/performance penalty for increase, 64, 128?

In my personal testing, I've been unable to observe a significant
performance impact (as Tom mentioned, I tried getting some profiling
data with gprof + pgbench, and found that increasing NAMEDATALEN made
things *faster*). Whether that is enough of an endorsement to make
the change for 7.3, I'm not sure...

> FUNC_MAX_ARGS - disk/performance penalty for increase, 24, 32?

Until someone takes the time to determine what the performance
implications of this change will be, I don't think we should
change this. Given that no one has done any testing, I'm not
convinced that there's a lot of demand for this anyway.

> Point-in-time recovery - ready for 7.3?
> Reindex/btree shrinkage - does reindex need work, can btree be shrunk?

I think both of these should probably wait for 7.4

> display locks - ready?
> Prepared statements - ready?

Both of these are ready, only trivial changes are required.

> Schema handling - ready? interfaces? client apps?

Do we want all client interfaces / admin apps to be aware of schemas in
time for beta 1, or is the plan to fix these during the beta cycle?

Cheers,

Neil

--
Neil Conway <neilconway(at)rogers(dot)com>
PGP Key ID: DB3C29FC

On Wed, Jul 31, 2002 at 02:01:43AM -0300, Marc G. Fournier wrote:
> add in 'fix pg_hba.conf / password issues' to that too :)

I doubt that will make 7.3 -- the proposals I've seen on this topic
require some reasonably complex additions to the authentication
system. We also still need to hash out which design we're going
to implement. Given that it's pretty esoteric, I'd prefer this
wait for 7.4

Cheers,

Neil

--
Neil Conway <neilconway(at)rogers(dot)com>
PGP Key ID: DB3C29FC

nconway(at)klamath(dot)dyndns(dot)org (Neil Conway) writes:
>> FUNC_MAX_ARGS - disk/performance penalty for increase, 24, 32?

> Until someone takes the time to determine what the performance
> implications of this change will be, I don't think we should
> change this. Given that no one has done any testing, I'm not
> convinced that there's a lot of demand for this anyway.

The OpenACS guys really really wanted larger FUNC_MAX_ARGS (I think
they had some 25-arg functions). And we do see questions about
increasing the limit fairly often on the lists. I suspect we could
bump it up to 32 at little cost --- but someone should run some
experiments to verify.

regards, tom lane

On Wed, 31 Jul 2002, Neil Conway wrote:

> On Wed, Jul 31, 2002 at 02:11:06AM -0300, Marc G. Fournier wrote:
> > On Wed, 31 Jul 2002, Tom Lane wrote:
> > > * libpqxx is not integrated into build process nor docs. It should
> > > be integrated or reversed out before beta.
> >
> > I've requestsed that Jeorgen(sp?) move this over to GBorg ... its
> > something that can, and should be, built seperately from the base
> > distribution, along with at least a dozen other things we have bloating
> > the distribution now :( but at least that one hasn't been integrated yet
> > ...
>
> Mentioning that on -hackers would have been nice -- I've spent a while
> this week hacking autoconf / Makefiles to integrate libpqxx...
>
> The problem I have with removing libpqxx is that libpq++ is a far
> inferior C++ interface. If we leave libpq++ as the only C++ interface
> distributed with PostgreSQL, there will be a tendancy for people
> using PostgreSQL & C++ to use the C++ support included with the
> distribution. Similarly, the Perl interface included with
> PostgreSQL is widely regarded as inferior to DBD::Pg.

Exactly what I mean ... we have *alot* of fat in the distribution ...
stuff that almost nobody uses ... or stuff that is generally inferior to
something else out there ...

What I *want* to see is a *base* server distribution vs the client side
code ... I *want* to be able to download libpq on a client machine to
install mod_php4, for example, without having to download everything else
that I'll never need ...

> If we're going to start removing interfaces, I'd vote for the removal of
> perl5 & libpq++ as well as libpqxx. We can add a section to the
> documentation listing the available language interfaces (for languages
> like Python, where several interfaces exist, this would probably be a
> good idea anyway).

Definitely ... python, tcl, pgaccess ... all of that needs to go ... they
are "specialty" stuff that, in some cases, have nothing to do with the
server itself ..

Anything that can build *from* libpq already being installed (except for
those that are required to admin the server (ie. psql, pg_dump, etc)
should be yanked and maintained outside of the distribution ... if nobody
is maintaining it, then obviously nobody is using it, so why carry it?

We have the environment now to keep the development 'centralized' through
GBorg, which also means that others can be provided with CVS access as
maintainers, if, for instance, Joergen(sp?) wishes to bring someone else
on board to help ...

On Wed, 31 Jul 2002, Tom Lane wrote:

> nconway(at)klamath(dot)dyndns(dot)org (Neil Conway) writes:
> > Mentioning that on -hackers would have been nice -- I've spent a while
> > this week hacking autoconf / Makefiles to integrate libpqxx...
>
> Marc's opinion is not the same thing as a done deal ;-) --- we still
> have to discuss this, and if someone's already doing the integration
> work I think that's an important factor.
>
> > If we're going to start removing interfaces, I'd vote for the removal of
> > perl5 & libpq++ as well as libpqxx.
>
> Agreed on that point. We shouldn't be promoting old, crufty interface
> libraries when there are better ones available.
>
> I would personally prefer to see libpqxx integrated now, and then we
> could plan to remove libpq++ in a release or two (after giving people
> a reasonable opportunity to switch over). If anyone still cares about
> libpq++ at that point, it could be given a home on gborg.
>
> One reason for wanting to integrate libpqxx is that I don't think we'll
> find out anything about its portability until we get a lot of people
> trying to build it. If it's a separate distro that won't happen quickly.

Who cares? Those that need a C++ interface will know where to find it,
and will report bugs that they have ... why should it be tested on every
platform when we *might* only have those on the Linux platform using it?

What happens if/when libpqxx becomes the 'old, crufty interface' and
something newer and shinier comes along? Where do we draw the line at
what is in the distribution? For instance, why pgaccess vs a platform
independent version of PgAdmin vs PHPPgAdmin? Hell, IMHO, PHPPgAdmin
would most likely be more useful as more sites out there are running PHP
then likely have TCL installed ... but someone that is using TCL/AolServer
would definitely think otherwise ...

By branching out the fat, we make it *easier* for someone to take on
development of it ... would libpqxx ever have been developed if Joergen
could have just worked on libpq++ in the first place, without having to
submit patches?

I really do not want to keep adding more users onto postgresql.org's
servers just because "hey, their interface is cool and useful so let's add
it into the main CVS repository and give them CVS access to save them
having to submit patches" when we have a fully functioning collaborative
development environment that gives them *more* then what we can give them
now ...

1. the ability to pull in their own group of developers / committers on a
per project basis
2. the ability to make releases *as required*, instead of having to wait
for us to do the next release

The benefit to us ... a much much smaller package of programs that have to
be maintained and tested and debugged before a release ...

Hell, how many packages do we currently have integrated with the whole
distribution that rely *nothing* on the server other then to be able to
use libpq to compile, that would benefit from being able to do releases?
If, for instance, the libpq++ interface gets patched up to fix a race
condition, or a vulnerability, the way things are right now, ppl have two
choices: wait for v7.3 to be released sometime in October, or upgrade to
the latest code via anon-cvs/cvsup ... and for package maintainers (rpm,
deb, pkg), they pretty much have no choice but to wait ...

Move libpq++ out as its own, independent project, and that patch would
force a quick packaging and release of the new code, which those same
package maintainers could jump on quickly and get out to *their* users ...

How many packages/interfaces do we have 'integrated' right now that rely
in no way on our release cycle *except* for the fact that they are
integrated?

The way we do things now made sense 7 years ago when we started out trying
to get it as visible to the masses as possible ... and when we *didn't*
have a clean/easy way to manage them externally ... it doesn't make any
sense to do anymore, and hasn't for a fair time now ...

On Wed, 31 Jul 2002, Neil Conway wrote:

> On Wed, Jul 31, 2002 at 02:01:43AM -0300, Marc G. Fournier wrote:
> > add in 'fix pg_hba.conf / password issues' to that too :)
>
> I doubt that will make 7.3 -- the proposals I've seen on this topic
> require some reasonably complex additions to the authentication
> system. We also still need to hash out which design we're going
> to implement. Given that it's pretty esoteric, I'd prefer this
> wait for 7.4

Then, the current changes *should* be removed, as we have no idea how many
sites out there we are going to break without that functionality ... I
know I personally have 200+ servers that will all break as soon as I move
to v7.3 with it as is :(

On Wed, Jul 31, 2002 at 02:08:33PM -0300, Marc G. Fournier wrote:
> On Wed, 31 Jul 2002, Tom Lane wrote:

> > One reason for wanting to integrate libpqxx is that I don't think we'll
> > find out anything about its portability until we get a lot of people
> > trying to build it. If it's a separate distro that won't happen quickly.
>
> Who cares? Those that need a C++ interface will know where to find it,
> and will report bugs that they have ... why should it be tested on every
> platform when we *might* only have those on the Linux platform using it?

This seems a bad argument. You can't say "we support interface xyz"
and never test it on anything except i80x86 Linux. Somebady comes
along and tries to make it go on Solaris, and it doesn't work: poof,
the cross-platform reputation that you and other have worked hard to
burnish goes away. Never mind that it's only a client library.

Besides, more generally, Postgres already has a reputation as being
difficult to install. The proposal to separate out all the
"non-basics" (I'm not even sure how one would draw that line: maybe a
server-only package and a client-library package run through GBorg?)
would mean that anyone wanting to do something moderately complicated
would have a yet higher hurdle. Isn't that a problem?

A

--
----
Andrew Sullivan 87 Mowat Avenue
Liberty RMS Toronto, Ontario Canada
<andrew(at)libertyrms(dot)info> M6K 3E3
+1 416 646 3304 x110

> Besides, more generally, Postgres already has a reputation as being
> difficult to install. The proposal to separate out all the
> "non-basics" (I'm not even sure how one would draw that line: maybe a
> server-only package and a client-library package run through GBorg?)
> would mean that anyone wanting to do something moderately complicated
> would have a yet higher hurdle. Isn't that a problem?

When you install freebsd or linux, is it a problem that all the perl modules
you need have to fetched from cpan ? why can't they call just be part of the
OS ?'
likewise with dns servers, samba, apache etc.. this is a bit of a stretched
example
but the point is the same.

Personall, I'd live to just be able to download the server with pg_dump psql
etc..

But that's just me for what it's worth.

jeff.

On Wed, 31 Jul 2002, Andrew Sullivan wrote:

> On Wed, Jul 31, 2002 at 02:08:33PM -0300, Marc G. Fournier wrote:
> > On Wed, 31 Jul 2002, Tom Lane wrote:
>
> > > One reason for wanting to integrate libpqxx is that I don't think we'll
> > > find out anything about its portability until we get a lot of people
> > > trying to build it. If it's a separate distro that won't happen quickly.
> >
> > Who cares? Those that need a C++ interface will know where to find it,
> > and will report bugs that they have ... why should it be tested on every
> > platform when we *might* only have those on the Linux platform using it?
>
> This seems a bad argument. You can't say "we support interface xyz"
> and never test it on anything except i80x86 Linux. Somebady comes
> along and tries to make it go on Solaris, and it doesn't work: poof,
> the cross-platform reputation that you and other have worked hard to
> burnish goes away. Never mind that it's only a client library.

This is my point, actually ... there are *two* things we should be
guarantee'ng to work cross-platform: the server and libpq ... (note that
with 'the server', I'm including the administrative commands and scripts,
like psql and initdb) ...

Take a look at libpq++ as a perfect example ... we've been distributing
that forever, but Tom himself states its 'old and crufty' ... but its also
the "officially supported version", so its what ppl generally use ...

We should be focusing on "the server", not the "clients" ...

Another point ... we have a load of stuff in contrib ... originally,
contrib was meant basically as a temporary storage while we decide if we
put it into the mainstream, and its grown into "if we have nowhere else to
put it, shove it in here and forget it" ... how many ppl know if all of
those that are in there even *work*? We know they compile, but do they
actually work?

> Besides, more generally, Postgres already has a reputation as being
> difficult to install. The proposal to separate out all the "non-basics"
> (I'm not even sure how one would draw that line: maybe a server-only
> package and a client-library package run through GBorg?) would mean that
> anyone wanting to do something moderately complicated would have a yet
> higher hurdle. Isn't that a problem?

Like what? I work at a local University and am slowly getting PgSQL used
for more and more things ... I have one server that is the database
server, but everything else connects to that ...

As it is now, I have to download the whole distribution, configure the
whole distribution, compile it and then install .. which, of course,
installs a bunch of stuff that I just don't need (initdb, psql, libpq++,
etc, etc) ... all I need is libpq.a ...

How many thousands of web sites out there don't offer PgSQL due to teh
hassle? Everyone is arguing 'why mysql vs pgsql?' ... if we had a simple
'libpq.tar.gz' that could be downloaded, nice and small, then we've just
made enabling PgSQL by default in mod_php4 brain dead ...

On Wed, Jul 31, 2002 at 02:36:43PM -0300, Jeff MacDonald wrote:
>
> When you install freebsd or linux, is it a problem that all the
> perl modules you need have to fetched from cpan ? why can't they
> call just be part of the OS ?'

Well, not just part of the OS, but part of Perl. And after all, Perl
_does_ include a fabulous variety of built-in modules.

> likewise with dns servers, samba, apache etc.. this is a bit of a stretched
> example
> but the point is the same.

Actually, the comparison is apt. There's a reason people suggest
using your distribution's PHP or Zope or what-have-you packages,
rather than installing from source: an inexperienced user with these
packages could easily spend several days trying to figure out all the
bits to install. Obviously, such people are new users, and a
learning curve is expected. But given recent hand-wringing about the
relative "mind-share" of Postgres &c., it seems perverse to make a
new user have to find out (probably by asking on a mailing list) that
basic stuff like client libraries are a whole separate package, which
needs to be dealt with separately. It _would_ be nice, though, to be
able to get just the client stuff for sure. And maybe the separation
is worth it; I just want to be sure that people know the effect on
users.

A

--
----
Andrew Sullivan 87 Mowat Avenue
Liberty RMS Toronto, Ontario Canada
<andrew(at)libertyrms(dot)info> M6K 3E3
+1 416 646 3304 x110

On Wed, Jul 31, 2002 at 03:11:40PM -0300, Marc G. Fournier wrote:

> hassle? Everyone is arguing 'why mysql vs pgsql?' ... if we had a simple
> 'libpq.tar.gz' that could be downloaded, nice and small, then we've just
> made enabling PgSQL by default in mod_php4 brain dead ...

Sorry, I think I wasn't making myself clear. I think that's a
splendid idea. But I'm not sure it's worth paying for it by making
users who want the whole thing download multiple packages. Maybe I'm
alone in thinking that, however, and it's not like I feel terribly
strongly about it.

A

--
----
Andrew Sullivan 87 Mowat Avenue
Liberty RMS Toronto, Ontario Canada
<andrew(at)libertyrms(dot)info> M6K 3E3
+1 416 646 3304 x110

Andrew Sullivan wrote:

> Sorry, I think I wasn't making myself clear. I think that's a
> splendid idea. But I'm not sure it's worth paying for it by making
> users who want the whole thing download multiple packages. Maybe I'm
> alone in thinking that, however, and it's not like I feel terribly
> strongly about it.

How much work is to make two packages - 'core' and 'complete'. Plus
additional package called 'util' = 'complete' minus 'core'.

> How many thousands of web sites out there don't offer PgSQL due to teh
> hassle? Everyone is arguing 'why mysql vs pgsql?' ... if we had a simple
> 'libpq.tar.gz' that could be downloaded, nice and small, then we've just
> made enabling PgSQL by default in mod_php4 brain dead ...

Case in point, I just installed FreeBSD 4.6 on a machine, i chose to install
mod_php from /stand/sysinstall.

It ofcourse installed php, with mysql as a dependency, i was annoyed, but
when
i looked at what was actually installed, it was just "mysql client".

The actually server did not get installed at all.

Jeff.

Tom Lane wrote:
> > Socket permissions - only install user can access db by default
>
> I do not agree with this goal.

OK, this is TODO item:

* Make single-user local access permissions the default by limiting
permissions on the socket file (Peter E)

Right now, we effectively install initdb as though we are creating a
world-writeable directory on the machine. (Sure, the directory is
locked down, but by setting PGUSER you can connect to the database as
anyone.) I don't know any other software that does this, and I can't
see how we can justify the current behavior.

Another idea is to change pg_hba.conf to not default to 'trust' but then
the installing user is going to have to choose a password.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Tom Lane wrote:
> > NAMEDATALEN - disk/performance penalty for increase, 64, 128?
> > FUNC_MAX_ARGS - disk/performance penalty for increase, 24, 32?
>
> At the moment I don't see a lot of solid evidence that increasing
> NAMEDATALEN has any performance penalty. Someone reported about
> a 10% slowdown on pgbench with NAMEDATALEN=128 ... but Neil Conway
> tried to reproduce the result, and got about a 10% *speedup*.
> Personally I think 10% is well within the noise spectrum for
> pgbench, and so it's difficult to claim that we have established
> any performance difference at all. I have not tried to measure
> FUNC_MAX_ARGS differences.

Yes, we need someone to benchmark both the NAMEDATALEN and FUNC_MAX_ARGS
to prove we are not causing performance problems. Once that is done,
the default limits can be easily increased. I was thinking 64 for
NAMEDATALEN and 32 for FUNC_MAX_ARGS, effectively doubling both.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Yes, we need someone to benchmark both the NAMEDATALEN and FUNC_MAX_ARGS
> to prove we are not causing performance problems. Once that is done,
> the default limits can be easily increased. I was thinking 64 for
> NAMEDATALEN and 32 for FUNC_MAX_ARGS, effectively doubling both.

The SQL spec says NAMEDATALEN shall be 128 (or at least 128, too lazy
to look). If we're gonna change it then I think we should really try
to go to 128.

regards, tom lane

On Wed, Jul 31, 2002 at 03:30:30PM -0400, Bruce Momjian wrote:
> Yes, we need someone to benchmark both the NAMEDATALEN and FUNC_MAX_ARGS
> to prove we are not causing performance problems. Once that is done,
> the default limits can be easily increased. I was thinking 64 for
> NAMEDATALEN and 32 for FUNC_MAX_ARGS, effectively doubling both.

If we're going to change NAMEDATALEN, we should probably set it to 128,
as that's what SQL99 requires.

Cheers,

Neil

--
Neil Conway <neilconway(at)rogers(dot)com>
PGP Key ID: DB3C29FC

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
> Socket permissions - only install user can access db by default
>>
>> I do not agree with this goal.

> OK, this is TODO item:

> * Make single-user local access permissions the default by limiting
> permissions on the socket file (Peter E)

Yes, I know what the TODO item says, and I disagree with it.

If we make the default permissions 700, then it's impossible to access
the database unless you run as the database owner. This is not a
security improvement --- it's more like claiming that a Linux system
would be more secure if you got rid of ordinary users and did all your
work as root. We should *not* encourage people to operate that way.
(It's certainly unworkable for RPM distributions anyway; only a user
who is hand-building a test installation under his own account would
possibly think that this is a useful default.)

I could see a default setup that made the permissions 770, allowing
access to anyone in the postgres group; that would at least bear some
slight resemblance to a workable production setup. However, this
assumes that the DBA has root privileges, else he'll not be able to
add/remove users from the postgres group. Also, on systems where users
all belong to the same "users" group, 770 isn't really better than 777.

The bottom line here is that there isn't any default protection setup
that is really widely useful. Everyone's got to adjust the thing to
fit their own circumstances. I'd rather see us spend more documentation
effort on pointing this out and explaining the alternatives, and not
think that we can solve the problem by making the default installation
so tight as to be useless.

regards, tom lane

> Another idea is to change pg_hba.conf to not default to 'trust' but then
> the installing user is going to have to choose a password.

I like this approach. Set it to password (or md5) on local, and force
the request of a password during initdb.

If for some reason they forget their password, they simply bump it to
trust on local for the 1 minute it takes to change it back.

>> Another idea is to change pg_hba.conf to not default to 'trust' but then
>> the installing user is going to have to choose a password.

Well, initdb already has an option to request a password. It would
perhaps make sense for initdb to alter the installed pg_hba.conf file
to select local md5 mode instead of local trust mode when this option is
specified.

> I like this approach. Set it to password (or md5) on local, and force
> the request of a password during initdb.

I don't like "forcing" people to do anything, especially not things that
aren't necessarily useful to them. On a single-user machine there is
no advantage to using database passwords.

regards, tom lane

Tom Lane wrote:
> > Point-in-time recovery - ready for 7.3?
>
> At the moment, it doesn't exist at all. If patches appear, we can
> review 'em, but right now there is nothing to debate.

Yes, I listed it just to keep it on the radar.

> > Win32 - timefame?
>
> I've seen nothing to make me think this will be ready for 7.3.

Same.

> > Schema handling - ready? interfaces? client apps?
>
> The backend will be ready (it's not quite yet). pg_dump is ready.
> psql is very definitely not ready, nor is pgaccess. I don't know the
> status for JDBC or ODBC; any comments? The other interface libraries
> probably don't care.

We should generate a list of subitems here.

> > Dependency - pg_dump auto-create dependencies for 7.2.X data?
>
> Huh?

Can we create table/sequence dependency linking on loads; same with
other dependencies? If not, we are going to have trouble with the
dependency code only working some times. This could be a serious
confusion for users. We coded some of our stuff assuming the linkage
will always be present, but a load from 7.2 may not have it. What are
the ramifications?

> > glibc and mktime() - fix?
>
> We need a fix for this. Dunno what to do about it.

I have proposed a fix of placing a fixed mktime earlier in the link line
but no one has supplied a fixed mktime for me.

> Other things on my radar screen:
>
> * I have about zero confidence in the recent tuple-header-size-reduction
> patches.

I have great confidence Manfred Koizar and his work. I know you want
some checks added to the code and he will do that when he returns. I
will mention this in the open items list.

improve macros in new tuple header code

>
> * pg_conversion stuff --- do we understand this thing's behavior under
> failure conditions? Does it work properly with namespaces and
> dependencies?

Seems Tatsuo says it is OK.

> * pg_dumpall probably ought to dump database privilege settings, also
> per-user and per-database GUC settings.

Added:

have pg_dumpall dump out db privilege and per-user/db settings

>
> * BeOS and QNX4 ports are busted.

Added:

fix BeOS and QNX4 ports
> * The whole area of which implicit coercions should be allowed is a
> serious problem that we have not spent enough time on. There are
> a number of cases in which CVS tip is clearly broken compared to
> prior releases.

Oh. I didn't know that. Added:

fix implicit type coercions that are worse
>
> * Bear Giles' SSL patches seem to be causing unhappiness in some
> quarters.

I believe it is only the interfaces/ssl directory I created from his
patch when I didn't know what to do with it. I wanted to remove it but
someone said it was good stuff and we should give the author until beta
to address it. Added to TODO:

remove interfaces/ssl if not improved

>
> * libpqxx is not integrated into build process nor docs. It should
> be integrated or reversed out before beta.

Added:

integrate or remove new libpqxx

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Neil Conway wrote:
> On Wed, Jul 31, 2002 at 03:30:30PM -0400, Bruce Momjian wrote:
> > Yes, we need someone to benchmark both the NAMEDATALEN and FUNC_MAX_ARGS
> > to prove we are not causing performance problems. Once that is done,
> > the default limits can be easily increased. I was thinking 64 for
> > NAMEDATALEN and 32 for FUNC_MAX_ARGS, effectively doubling both.
>
> If we're going to change NAMEDATALEN, we should probably set it to 128,
> as that's what SQL99 requires.

OK, updated to show only 128. Do we need more performance testing? I
am unclear on that.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Christopher Kings-Lynne wrote:
> > > Dependency - pg_dump auto-create dependencies for 7.2.X data?
> >
> > Huh?
>
> Taking a bunch of CREATE CONSTRAINT TRIGGERS and turning them into the
> proper pg_constraint entries...

Description updated:

Dependency - have pg_dump auto-create dependencies when loading
7.2.X data?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Thomas Lockhart wrote:
> ...
> > I agree that if we could quickly come to a resolution about how this
> > ought to work, there's plenty of time to go off and implement it. But
> > (1) we failed to come to a consensus before, so I'm not optimistic
> > than one will suddenly emerge now; (2) we've got a ton of other issues
> > that we *need* to deal with before beta. This one does not strike me
> > as a must-fix, and so I'm loathe to spend much development time on it
> > when there are so many open issues.
>
> afaict someone else volunteered to do the work. There is no lack of
> consensus that this is a useful feature, at least among those who take
> responsibility to package PostgreSQL for particular platforms. How about
> letting them specify the requirements and if an acceptable solution
> emerges soon, we'll have it for 7.3...

Added to open items:

allow specification of configuration files in a different directory

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Neil Conway wrote:
> On Tue, Jul 30, 2002 at 11:50:38PM -0400, Bruce Momjian wrote:
> > NAMEDATALEN - disk/performance penalty for increase, 64, 128?
>
> In my personal testing, I've been unable to observe a significant
> performance impact (as Tom mentioned, I tried getting some profiling
> data with gprof + pgbench, and found that increasing NAMEDATALEN made
> things *faster*). Whether that is enough of an endorsement to make
> the change for 7.3, I'm not sure...

OK, do we need to test further or just bump it to 128?

> > FUNC_MAX_ARGS - disk/performance penalty for increase, 24, 32?
>
> Until someone takes the time to determine what the performance
> implications of this change will be, I don't think we should
> change this. Given that no one has done any testing, I'm not
> convinced that there's a lot of demand for this anyway.

I am going to list only 32 as an option. I think it needs to be at
least that high for OpenACS.

> > Point-in-time recovery - ready for 7.3?
> > Reindex/btree shrinkage - does reindex need work, can btree be shrunk?
>
> I think both of these should probably wait for 7.4

Probably. We will just keep them on the radar.

> > Schema handling - ready? interfaces? client apps?
>
> Do we want all client interfaces / admin apps to be aware of schemas in
> time for beta 1, or is the plan to fix these during the beta cycle?

Ideally, before beta or people can't really test them.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Marc G. Fournier wrote:
> On Wed, 31 Jul 2002, Neil Conway wrote:
>
> > On Wed, Jul 31, 2002 at 02:01:43AM -0300, Marc G. Fournier wrote:
> > > add in 'fix pg_hba.conf / password issues' to that too :)
> >
> > I doubt that will make 7.3 -- the proposals I've seen on this topic
> > require some reasonably complex additions to the authentication
> > system. We also still need to hash out which design we're going
> > to implement. Given that it's pretty esoteric, I'd prefer this
> > wait for 7.4
>
> Then, the current changes *should* be removed, as we have no idea how many
> sites out there we are going to break without that functionality ... I
> know I personally have 200+ servers that will all break as soon as I move
> to v7.3 with it as is :(

OK, I have thought about this. First, a possible solution would be to
have a GUC variable that prepends the dbname to all username
specifications, so the username becomes dbname.username. When you
CREATE USER "test", it actually does CREATE USER "dbname.test". Same
with ALTER/DROP user and lookups in pg_hba.conf and authentication.
Basically it gives us a per-db user namespace. Only the superuser has a
non-db qualified name. (Actually, createuser script would fail because
it connects only to template1. You would have to use psql and CREATE
USER. Probably other things would fail too.)

As for 7.3, maybe we can get that done in time of everyone likes it. If
we can't, what do we do? Do we re-add the secondary password file stuff
that most people don't like? My big question is how many other
PostgreSQL users figured out they could use the secondary password file
for username/db restrictions? I never thought of it myself. Maybe I
should ask on general.

Marc, you do have a workaround for 7.3 using your IP's, right, or is
there a problem with the password having to be the same for different
hosts with the same username? If Marc is the only one, and he has a
workaround, we may just go ahead and leave it for 7.4.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Added to open items list:

handle lack of secondary passwords

---------------------------------------------------------------------------

Marc G. Fournier wrote:
>
> add in 'fix pg_hba.conf / password issues' to that too :)
>
> On Tue, 30 Jul 2002, Bruce Momjian wrote:
>
> >
> > Here are the open items for 7.3. We have one more month to address them
> > before beta.
> >
> > ---------------------------------------------------------------------------
> >
> > P O S T G R E S Q L
> >
> > 7 . 3 O P E N I T E M S
> >
> >
> > Current at ftp://candle.pha.pa.us/pub/postgresql/open_items.
> >
> > Source Code Changes
> > -------------------
> > Socket permissions - only install user can access db by default
> > unix_socket_permissions in postgresql.conf
> > NAMEDATALEN - disk/performance penalty for increase, 64, 128?
> > FUNC_MAX_ARGS - disk/performance penalty for increase, 24, 32?
> > Point-in-time recovery - ready for 7.3?
> > Allow easy display of usernames in a group (pg_hba.conf uses groups now)
> > Reindex/btree shrinkage - does reindex need work, can btree be shrunk?
> > DROP COLUMN - ready?
> > CLUSTER - ready?
> > display locks - ready?
> > Win32 - timefame?
> > Prepared statements - ready?
> > Schema handling - ready? interfaces? client apps?
> > Dependency - pg_dump auto-create dependencies for 7.2.X data?
> > glibc and mktime() - fix?
> > Functions Returning Sets - done?
> > ecpg and bison issues - solved?
> >
> >
> > Documentation Changes
> > ---------------------
> >
> >
> > --
> > Bruce Momjian | http://candle.pha.pa.us
> > pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
> > + If your life is a hard drive, | 830 Blythe Avenue
> > + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 3: if posting/reading through Usenet, please send an appropriate
> > subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> > message can get through to the mailing list cleanly
> >
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

> > psql is very definitely not ready, nor is pgaccess.

I could not really trace who said this.

To my understanding nobody is currently testing how pgaccess is dealing
with 7.3 Am I wrong?

Most problems we try to address now are related to pgaccess working on
most platforms (Brett fights with the dlls, there are some Mac OS X
efforts) and improve the usability (help upon connection failure, etc.)

There are many new features people write, but this has not much to do
with 7.3 in a direct way.

Now in addition to the bugzilla and the developers(at)pgaccess(dot)org mailing
list, there is also a wiki

http://www.pgaccess.org/wiki/

as a channel for filing ideas and wishes.

Please, feel free to use it.

Iavor

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> > Socket permissions - only install user can access db by default
> >>
> >> I do not agree with this goal.
>
> > OK, this is TODO item:
>
> > * Make single-user local access permissions the default by limiting
> > permissions on the socket file (Peter E)
>
> Yes, I know what the TODO item says, and I disagree with it.
>
> If we make the default permissions 700, then it's impossible to access
> the database unless you run as the database owner. This is not a
> security improvement --- it's more like claiming that a Linux system
> would be more secure if you got rid of ordinary users and did all your
> work as root. We should *not* encourage people to operate that way.
> (It's certainly unworkable for RPM distributions anyway; only a user
> who is hand-building a test installation under his own account would
> possibly think that this is a useful default.)

I hope they would loosen the default in postgresql.conf rather than
having everyone come in as the same user. By the time you create new
user accounts, it is trivial to modify postgresql.conf.

> I could see a default setup that made the permissions 770, allowing
> access to anyone in the postgres group; that would at least bear some
> slight resemblance to a workable production setup. However, this
> assumes that the DBA has root privileges, else he'll not be able to
> add/remove users from the postgres group. Also, on systems where users
> all belong to the same "users" group, 770 isn't really better than 777.

Yes, groups are nice, but in most cases with a group 'users', it is the
same as world-writable.

> The bottom line here is that there isn't any default protection setup
> that is really widely useful. Everyone's got to adjust the thing to
> fit their own circumstances. I'd rather see us spend more documentation
> effort on pointing this out and explaining the alternatives, and not
> think that we can solve the problem by making the default installation
> so tight as to be useless.

I think we are much safer shipping as secure and asking people to loosen
it if they want wider access. I can imagine a Bugtrack item for
PostgreSQL where they report we ship wide-open for local users. They
have already reported we don't encrypt our passwords, and we are dealing
with that. You can say that we tell people to change the default, but
if we install that way, they have a legitimate grip, and PostgreSQL has a
perception problem.

The default unix permissions are world-readable, owner-writable. We
ship with world-read/write. I know of _no_ other software that does
that and I can't see how we get away with it. I will also add that I am
the biggest proponent of tightening things up and on one else seems to
be as concerned about it. I am not sure why.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Tom Lane wrote:
> >> Another idea is to change pg_hba.conf to not default to 'trust' but then
> >> the installing user is going to have to choose a password.
>
> Well, initdb already has an option to request a password. It would
> perhaps make sense for initdb to alter the installed pg_hba.conf file
> to select local md5 mode instead of local trust mode when this option is
> specified.
>
> > I like this approach. Set it to password (or md5) on local, and force
> > the request of a password during initdb.
>
> I don't like "forcing" people to do anything, especially not things that
> aren't necessarily useful to them. On a single-user machine there is
> no advantage to using database passwords.

Yes, on a single-user machine, socket permissions are a better approach.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Andrew Sullivan wrote:
> Actually, the comparison is apt. There's a reason people suggest
> using your distribution's PHP or Zope or what-have-you packages,
> rather than installing from source: an inexperienced user with these
> packages could easily spend several days trying to figure out all the
> bits to install. Obviously, such people are new users, and a
> learning curve is expected. But given recent hand-wringing about the
> relative "mind-share" of Postgres &c., it seems perverse to make a
> new user have to find out (probably by asking on a mailing list) that
> basic stuff like client libraries are a whole separate package, which
> needs to be dealt with separately. It _would_ be nice, though, to be
> able to get just the client stuff for sure. And maybe the separation
> is worth it; I just want to be sure that people know the effect on
> users.

I have already provide Marc with a script needed to make an
interfaces-only tarball. I was about 1/10th the size of the full
tarball.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Tom Lane writes:

> > Socket permissions - only install user can access db by default
>
> I do not agree with this goal.

It is my understanding that there is currently a lot of criticism that the
default setup is open to all local users. This is nearly the same as
having the data area files themselves world-writable by default.

Maybe changing the default socket permissions isn't the appropriate
measure, but I feel something ought to be done.

--
Peter Eisentraut peter_e(at)gmx(dot)net

> Mentioning that on -hackers would have been nice -- I've spent a while
> this week hacking autoconf / Makefiles to integrate libpqxx...
>
> The problem I have with removing libpqxx is that libpq++ is a far
> inferior C++ interface. If we leave libpq++ as the only C++ interface
> distributed with PostgreSQL, there will be a tendancy for people
> using PostgreSQL & C++ to use the C++ support included with the
> distribution. Similarly, the Perl interface included with
> PostgreSQL is widely regarded as inferior to DBD::Pg.

I think that if someone is actually working on libpqxx integration - then
yeah, leave it in...

Chris

On Wed, 31 Jul 2002, Bruce Momjian wrote:

> OK, I have thought about this. First, a possible solution would be to
> have a GUC variable that prepends the dbname to all username
> specifications, so the username becomes dbname.username. When you
> CREATE USER "test", it actually does CREATE USER "dbname.test". Same
> with ALTER/DROP user and lookups in pg_hba.conf and authentication.
> Basically it gives us a per-db user namespace. Only the superuser has a
> non-db qualified name. (Actually, createuser script would fail because
> it connects only to template1. You would have to use psql and CREATE
> USER. Probably other things would fail too.)

Sounds like a good solution that eliminates Tom's idea of going with
'local to database' pg_shadow files ... I like it ...

> As for 7.3, maybe we can get that done in time of everyone likes it.
> If we can't, what do we do? Do we re-add the secondary password file
> stuff that most people don't like? My big question is how many other
> PostgreSQL users figured out they could use the secondary password file
> for username/db restrictions? I never thought of it myself. Maybe I
> should ask on general.

How many ppl that aren't subscribed to any of the lists are using this and
are going to get burned royal when they upgrade to v7.3 without
understanding it is gone?

> Marc, you do have a workaround for 7.3 using your IP's, right, or is
> there a problem with the password having to be the same for different
> hosts with the same username? If Marc is the only one, and he has a
> workaround, we may just go ahead and leave it for 7.4.

No, unfortunately, I have at least one specific application that needs
this :( IPs help reduce the impact of losing this, but with the growing
difficultly and cost of acquiring new IPs, the IPs don't help much either
:(

> > As for 7.3, maybe we can get that done in time of everyone likes it.
> > If we can't, what do we do? Do we re-add the secondary password file
> > stuff that most people don't like? My big question is how many other
> > PostgreSQL users figured out they could use the secondary password file
> > for username/db restrictions? I never thought of it myself. Maybe I
> > should ask on general.
>
> How many ppl that aren't subscribed to any of the lists are using this and
> are going to get burned royal when they upgrade to v7.3 without
> understanding it is gone?

I agree with Marc here - compatibility in this area might just be very
important (compared with some other lesser areas of incompatibility...)

Chris

On Wed, Jul 31, 2002 at 05:05:35PM -0400, Bruce Momjian wrote:
> OK, I have thought about this. First, a possible solution would be to
> have a GUC variable that prepends the dbname to all username
> specifications, so the username becomes dbname.username. When you
> CREATE USER "test", it actually does CREATE USER "dbname.test". Same
> with ALTER/DROP user and lookups in pg_hba.conf and authentication.
> Basically it gives us a per-db user namespace. Only the superuser has a
> non-db qualified name.

What about the following situation:

- 3 databases: 'devel', 'staging', and 'production'

- one user, 'httpd', which needs access to all 3 databases but
doesn't own any of them

- I create the 'httpd' user when I'm connected to, say, template1

- I issue a command that changes the httpd user in some way (e.g.
drops the user, alters the user, etc.) -- what happens?

Also, what happens if I enable the GUC var, create a bunch of different
users/databases, and then disable it again?

Cheers,

Neil

--
Neil Conway <neilconway(at)rogers(dot)com>
PGP Key ID: DB3C29FC

Neil Conway wrote:
> On Wed, Jul 31, 2002 at 05:05:35PM -0400, Bruce Momjian wrote:
> > OK, I have thought about this. First, a possible solution would be to
> > have a GUC variable that prepends the dbname to all username
> > specifications, so the username becomes dbname.username. When you
> > CREATE USER "test", it actually does CREATE USER "dbname.test". Same
> > with ALTER/DROP user and lookups in pg_hba.conf and authentication.
> > Basically it gives us a per-db user namespace. Only the superuser has a
> > non-db qualified name.
>
> What about the following situation:
>
> - 3 databases: 'devel', 'staging', and 'production'
>
> - one user, 'httpd', which needs access to all 3 databases but
> doesn't own any of them
>
> - I create the 'httpd' user when I'm connected to, say, template1
>
> - I issue a command that changes the httpd user in some way (e.g.
> drops the user, alters the user, etc.) -- what happens?

I am going to require the admin to prepend the dbname. GUC controls
whether authentication/username map from just the client-supplied
username, or the client username prepended with the dbname.

>
> Also, what happens if I enable the GUC var, create a bunch of different
> users/databases, and then disable it again?

You swap back and forth between users with prepended dbnames and those
withouth.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

On Wed, Jul 31, 2002 at 11:40:43PM -0400, Bruce Momjian wrote:
> > Also, what happens if I enable the GUC var, create a bunch of different
> > users/databases, and then disable it again?
>
> You swap back and forth between users with prepended dbnames and those
> withouth.

And if I've created the user before I enabled the GUC var, how does
authentication work?

Cheers,

Neil

--
Neil Conway <neilconway(at)rogers(dot)com>
PGP Key ID: DB3C29FC

Neil Conway wrote:
> On Wed, Jul 31, 2002 at 11:40:43PM -0400, Bruce Momjian wrote:
> > > Also, what happens if I enable the GUC var, create a bunch of different
> > > users/databases, and then disable it again?
> >
> > You swap back and forth between users with prepended dbnames and those
> > withouth.
>
> And if I've created the user before I enabled the GUC var, how does
> authentication work?

User creation will not be effected. You have to prepend the dbname
yourself. This will _now_ only effect modification of the user name as
passed from the client.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

On Thu, 2002-08-01 at 02:05, Bruce Momjian wrote:
> Marc G. Fournier wrote:
> > On Wed, 31 Jul 2002, Neil Conway wrote:
> >
> > > On Wed, Jul 31, 2002 at 02:01:43AM -0300, Marc G. Fournier wrote:
> > > > add in 'fix pg_hba.conf / password issues' to that too :)
> > >
> > > I doubt that will make 7.3 -- the proposals I've seen on this topic
> > > require some reasonably complex additions to the authentication
> > > system. We also still need to hash out which design we're going
> > > to implement. Given that it's pretty esoteric, I'd prefer this
> > > wait for 7.4
> >
> > Then, the current changes *should* be removed, as we have no idea how many
> > sites out there we are going to break without that functionality ... I
> > know I personally have 200+ servers that will all break as soon as I move
> > to v7.3 with it as is :(
>
> OK, I have thought about this. First, a possible solution would be to
> have a GUC variable that prepends the dbname to all username
> specifications, so the username becomes dbname.username.

When I first read Marc's post about this I also thought that the users
were partitioned by database, but further reading revealed that tis was
not the case - actually they were partitioned by _a_group_of_databases_,
as each of his virtual hosts accesses on _at_least_ one but possibly
more databases using the same user (bruce ;).

So we would need some sort of database groups that share the same users.

We have to do something like this:

real_user_name = mk_real_user_name(username,dbname)

which uses some mapping table to find the real user that is trying to
connect to the database.

This name mangling should be done at connect time and kept out of
database, where each users name should always be fully resolved
(bob(at)accounting(dot)acme(dot)com).

This may require raising the length of NAME type to be backwards
compatible. Or we migth just add USEDOMAIN column to uniquely identify
the user. so the above user would still have usename=bob but also
usedomain="accounting.acme.com".

-----------
Hannu

> > * libpqxx is not integrated into build process nor docs. It should
> > be integrated or reversed out before beta.
> I've requestsed that Jeorgen(sp?) move this over to GBorg ... its
> something that can, and should be, built seperately from the base
> distribution, along with at least a dozen other things we have bloating
> the distribution now :( but at least that one hasn't been integrated yet
> ...

Actually, I'm not sure we should target one particular feature to be
left out, unless we have folks who are willing to do the planning,
design, and implementation of a "sliced and diced PostgreSQL" in a
consistant and solid way.

Until we have folks who are excited enough about it to plan it out and
do the work, piecemeal rejection of components is not leading to a more
solid product.

The developers have made the commitment to have consistant and
functional builds across all packages in the main distro. We have no
mechanisms to do the same if the sources are coming from a bunch of
different areas.

Frankly, a 9MB tarball is not in the bloat category in my book. Ask me
about the CORBA package I use that takes 3.5GB to build!!

- Thomas

Thomas Lockhart <lockhart(at)fourpalms(dot)org> writes:
> Until we have folks who are excited enough about it to plan it out and
> do the work, piecemeal rejection of components is not leading to a more
> solid product.

I'm lukewarm about whether to actually do the split or not ... but for
sure I agree with Thomas' point here. We need a plan and careful
implementation, or a split-up will just make life worse.

Stuff that is in the tree tends to get maintained in passing. For
example, I've got some changes to contrib/dblink/ in my in-progress
version of Chris' DROP COLUMN patch, because a grep for references
to rel->rd_att turned it up. If dblink weren't in our CVS it'd have
been broken by DROP COLUMN, and who knows whether we'd catch that
during beta? I realize that Marc wasn't proposing splitting off any
server-side code, but I still want to tread carefully about breaking
up the codebase.

regards, tom lane

On Thu, 1 Aug 2002, Tom Lane wrote:

> Thomas Lockhart <lockhart(at)fourpalms(dot)org> writes:
> > Until we have folks who are excited enough about it to plan it out and
> > do the work, piecemeal rejection of components is not leading to a more
> > solid product.
>
> I'm lukewarm about whether to actually do the split or not ... but for
> sure I agree with Thomas' point here. We need a plan and careful
> implementation, or a split-up will just make life worse.
>
> Stuff that is in the tree tends to get maintained in passing. For
> example, I've got some changes to contrib/dblink/ in my in-progress
> version of Chris' DROP COLUMN patch, because a grep for references
> to rel->rd_att turned it up. If dblink weren't in our CVS it'd have
> been broken by DROP COLUMN, and who knows whether we'd catch that
> during beta? I realize that Marc wasn't proposing splitting off any
> server-side code, but I still want to tread carefully about breaking
> up the codebase.

Okay, well, the way I'm working it through right now, I'm doing it in such
a way that unless you go mucking in the repository directly, it will be
transparent to the coders, as well as to the distribution as a whole ...

In fact, based on a comment that Thomas made in another email, I'll even
fix up the whole 'cvs checkout pgsql' thing so that that goes back to its
previous incarnation of pulling everything, instead of needing to do
pgsql-all ...

So, from the 'client side', y'all will still see "everything as one big
package", while from the 'server side', I'll have the seperate modules
taht can be packaged independently ...

Next, unless Peter knows how to do it already, I've gotta learn to make
configure more intelligent, so that for all intents, the "pieces" look
like one package when building, not just when coding ...

Bruce Momjian dijo:

> Here are the open items for 7.3. We have one more month to address them
> before beta.

> CLUSTER - ready?

I'm just back. I'll have a look at the problem with the patch and
resubmit.

--
Alvaro Herrera (<alvherre[a]atentus.com>)
"Es filosofo el que disfruta con los enigmas" (G. Coli)

Le Mercredi 31 Juillet 2002 05:50, Bruce Momjian a écrit :
> Here are the open items for 7.3. We have one more month to address them
> before beta.

Is CREATE OR REPLACE VIEW on the list?

"Marc G. Fournier" <scrappy(at)hub(dot)org> writes:
>> I realize that Marc wasn't proposing splitting off any
>> server-side code, but I still want to tread carefully about breaking
>> up the codebase.

> Okay, well, the way I'm working it through right now, I'm doing it in such
> a way that unless you go mucking in the repository directly, it will be
> transparent to the coders, as well as to the distribution as a whole ...

> In fact, based on a comment that Thomas made in another email, I'll even
> fix up the whole 'cvs checkout pgsql' thing so that that goes back to its
> previous incarnation of pulling everything, instead of needing to do
> pgsql-all ...

Okay, that works for me --- that makes it just a packaging issue, and
not something that will hide stuff from people who want to look through
the whole tree.

regards, tom lane

On Thu, 1 Aug 2002, Tom Lane wrote:

> "Marc G. Fournier" <scrappy(at)hub(dot)org> writes:
> >> I realize that Marc wasn't proposing splitting off any
> >> server-side code, but I still want to tread carefully about breaking
> >> up the codebase.
>
> > Okay, well, the way I'm working it through right now, I'm doing it in such
> > a way that unless you go mucking in the repository directly, it will be
> > transparent to the coders, as well as to the distribution as a whole ...
>
> > In fact, based on a comment that Thomas made in another email, I'll even
> > fix up the whole 'cvs checkout pgsql' thing so that that goes back to its
> > previous incarnation of pulling everything, instead of needing to do
> > pgsql-all ...
>
> Okay, that works for me --- that makes it just a packaging issue, and
> not something that will hide stuff from people who want to look through
> the whole tree.

Actually, it makes it a 'storage' issue on the CVS server itself, but
makes creating various packages easier ... I've pop'd off an email to the
libpqxx configure guys to get their standalone configure issues fixed (try
running autogen.sh), after which I want to look into 'calling' the
standalone configure from the global one if --enable-libpqxx is called
(which we can later default to 'on' if that should become the default) ...

Hannu Krosing <hannu(at)tm(dot)ee> writes:
> This name mangling should be done at connect time and kept out of
> database, where each users name should always be fully resolved
> (bob(at)accounting(dot)acme(dot)com).

I really like Hannu's approach to this. It seems to solve Marc's
problem with a very simple, easily understood, easily implemented
feature. All we need is a postmaster configuration parameter that
(when TRUE) causes the postmaster to convert the passed username
into 'username(at)databasename' before looking it up in pg_shadow.

(Actually, what I'd prefer it do is try first for username, and
then username(at)databasename if plain username isn't found.)

With this approach, we have an underlying mechanism that supports
installation-wide usernames, same as before, but with the flip of
a switch you can configure the system to support per-database
usernames. It's not fancy, maybe, but it will get the job done
with an appropriate amount of effort.

We've had several proposals in this thread for complicated extensions
to the user naming mechanism. I think that's overdesigning the feature,
because we have *no* examples of real-world need for such things except
for Marc's situation. Let's keep it simple until we see real use cases
that can drive the design of something fancy.

> This may require raising the length of NAME type to be backwards
> compatible.

Right, but we're planning to do that anyway.

regards, tom lane

Jean-Michel POURE wrote:
> Le Mercredi 31 Juillet 2002 05:50, Bruce Momjian a ?crit :
> > Here are the open items for 7.3. We have one more month to address them
> > before beta.
>
> Is CREATE OR REPLACE VIEW on the list?

No. It can still be done, but no one is working on it.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

On Wednesday 31 July 2002 04:56 pm, Bruce Momjian wrote:
> Thomas Lockhart wrote:
>> Tom Lane wrote:
> > > I agree that if we could quickly come to a resolution about how this
> > > ought to work, there's plenty of time to go off and implement it.

> > afaict someone else volunteered to do the work. There is no lack of
> > consensus that this is a useful feature, at least among those who take
> > responsibility to package PostgreSQL for particular platforms. How about
> > letting them specify the requirements and if an acceptable solution
> > emerges soon, we'll have it for 7.3...

> Added to open items:

> allow specification of configuration files in a different directory

Thanks Bruce.

I am going to review the previous thread and attempt to distill what can be
done. I will then post a summary of what I found, with potential commentary.
If a consensus is reached, I'd like to see the feature in 7.3.

Peter had mentioned it as well; might want to see if he has done anything as
yet with it.

That being said, a patch exists for 7.2beta to effect a version of this
change. I will also review this patch as I can and see what will be required
to make this work in CURRENT.

IMO, the key is that if the switch is not specified the current behavior is
default. If specified, it will do its thing.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Tom Lane wrote:
> Hannu Krosing <hannu(at)tm(dot)ee> writes:
> > This name mangling should be done at connect time and kept out of
> > database, where each users name should always be fully resolved
> > (bob(at)accounting(dot)acme(dot)com).
>
> I really like Hannu's approach to this. It seems to solve Marc's
> problem with a very simple, easily understood, easily implemented
> feature. All we need is a postmaster configuration parameter that
> (when TRUE) causes the postmaster to convert the passed username
> into 'username(at)databasename' before looking it up in pg_shadow.

Yes, that is how the patch I submitted last night does it.

> (Actually, what I'd prefer it do is try first for username, and
> then username(at)databasename if plain username isn't found.)

Yes, that would be very easy to do _except_ for pg_hba.conf which does a
first-match for username. We could get into trouble there by trying two
versions of the same name. Comments?

> With this approach, we have an underlying mechanism that supports
> installation-wide usernames, same as before, but with the flip of
> a switch you can configure the system to support per-database
> usernames. It's not fancy, maybe, but it will get the job done
> with an appropriate amount of effort.
>
> We've had several proposals in this thread for complicated extensions
> to the user naming mechanism. I think that's overdesigning the feature,
> because we have *no* examples of real-world need for such things except
> for Marc's situation. Let's keep it simple until we see real use cases
> that can drive the design of something fancy.

Agreed.

>
> > This may require raising the length of NAME type to be backwards
> > compatible.
>
> Right, but we're planning to do that anyway.

Yes, but that requires a protocol change, which we don't want to do for
7.3. My fix is to just extend the username on the server side and
append the dbname if the switch is on.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Marc G. Fournier wrote:
> So, from the 'client side', y'all will still see "everything as one big
> package", while from the 'server side', I'll have the seperate modules
> taht can be packaged independently ...

Marc, how are you dealing with libpq's access to the server include
files?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

On Thu, 1 Aug 2002, Bruce Momjian wrote:

> Marc G. Fournier wrote:
> > So, from the 'client side', y'all will still see "everything as one big
> > package", while from the 'server side', I'll have the seperate modules
> > taht can be packaged independently ...
>
> Marc, how are you dealing with libpq's access to the server include
> files?

I haven't touched libpq yet ... I'm talking with the libpqxx guys right
now concerning getting the standalone libpqxx to work, and will be working
on figuring out how to get the 'master configure' to make use of the
standalone configure once that is fixed ... I want to get one module to
work cleanly before looking at any others ...

Marc G. Fournier wrote:
> On Thu, 1 Aug 2002, Bruce Momjian wrote:
>
> > Marc G. Fournier wrote:
> > > So, from the 'client side', y'all will still see "everything as one big
> > > package", while from the 'server side', I'll have the seperate modules
> > > taht can be packaged independently ...
> >
> > Marc, how are you dealing with libpq's access to the server include
> > files?
>
> I haven't touched libpq yet ... I'm talking with the libpqxx guys right
> now concerning getting the standalone libpqxx to work, and will be working
> on figuring out how to get the 'master configure' to make use of the
> standalone configure once that is fixed ... I want to get one module to
> work cleanly before looking at any others ...

But isn't libpq++ just going to be part of interfaces?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

On Thu, 2002-08-01 at 16:17, Tom Lane wrote:
> Hannu Krosing <hannu(at)tm(dot)ee> writes:
> > This name mangling should be done at connect time and kept out of
> > database, where each users name should always be fully resolved
> > (bob(at)accounting(dot)acme(dot)com).
>
> I really like Hannu's approach to this. It seems to solve Marc's
> problem with a very simple, easily understood, easily implemented
> feature. All we need is a postmaster configuration parameter that
> (when TRUE) causes the postmaster to convert the passed username
> into 'username(at)databasename' before looking it up in pg_shadow.
>
> (Actually, what I'd prefer it do is try first for username, and
> then username(at)databasename if plain username isn't found.)

This should not really be @databasename, but rather a @domainname as
Mark does in fact want to use the same user from some virtual host
(==domain) for more than one database sometimes.

Using databasename as a domainname is just the quickest way to resolve
the domainname if no more info about it is given.

Thinking of the @xxx part as a domainname and not tying it to
databasename would be beneficial in case we later want to use other
kinds of domains (like NT, DNS/mail, YP or Kerberos domains for example)

If need arises we could later split out the @xxx part to "usedomain"
field and perhaps also add "usedomainkind" field in order to manage that
info in databse instead of pg_hba.conf.

-----------------
Hannu

On Thu, 1 Aug 2002, Bruce Momjian wrote:

> Marc G. Fournier wrote:
> > On Thu, 1 Aug 2002, Bruce Momjian wrote:
> >
> > > Marc G. Fournier wrote:
> > > > So, from the 'client side', y'all will still see "everything as one big
> > > > package", while from the 'server side', I'll have the seperate modules
> > > > taht can be packaged independently ...
> > >
> > > Marc, how are you dealing with libpq's access to the server include
> > > files?
> >
> > I haven't touched libpq yet ... I'm talking with the libpqxx guys right
> > now concerning getting the standalone libpqxx to work, and will be working
> > on figuring out how to get the 'master configure' to make use of the
> > standalone configure once that is fixed ... I want to get one module to
> > work cleanly before looking at any others ...
>
> But isn't libpq++ just going to be part of interfaces?

Huh? Each module is to be designed as a standalone project/distribution
... in order to appease those that don't feel that the change is worth it,
I'm making, essentially, a 'meta-module' that will pull in the seperate
modules into what you've all gotten used to from a development standpoint
...

If you checkout pgsql, you will see what you are used to seeing, locally
stored in pgsql

If you checkout libpqxx, you will just get libpqxx, locally stored in
libpqxx

If you checkout interfaces, you will get all of the interfaces listed in a
'meta module' consisting of the various interfaces, locally stored in a
directory of pgsql/src/interfaces/*

For those that are used to cheking out pgsql, continue to do so ... for
ppl like Jergeon(sp?), he will checkout libpqxx itself and work on it as
if it were a standalone project, but when we package up pgsql, it will get
pulled in along with everything else, so that for those that have nothing
better to do then download everyhting and the kitchen sink, they can ...

At the same time as the distribution is made, a libpqxx.tar.gz will be
created, that will be a self-contained source tree for just that, so that
those doing ports on the *BSDs or rpms/etc on Linux have pretty much
pre-made distros that they don't have to slice and dice (ie. for FreeBSD,
you'd do something like go into /usr/ports/database/pgsql-libpqxx, type
'make' and it would automatically go out, download libpq.tar.gz, install
it as a dependency and then grab and install the libpqxx file ... if you
had already installed libpq previously, for mod_php4, for example, then it
would just download and install the libpqxx tar file) ...

Unless I break something along the way, as far as you are concerned,
nothing has changed ... just keep checking out pgsql as you always have
... but for those of us that pay for our bandwidth by the byte, we'll now
be able to download *only* what we require, saving us both time and money
...

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
>> (Actually, what I'd prefer it do is try first for username, and
>> then username(at)databasename if plain username isn't found.)

> Yes, that would be very easy to do _except_ for pg_hba.conf which does a
> first-match for username. We could get into trouble there by trying two
> versions of the same name. Comments?

Hm. I think we'd have to switch around the order of stuff so that we
look at the flat-file copy of pg_shadow first. Then we'd know which
flavor of name we have, and we can proceed with the pg_hba match.

The reason it's worth doing this is that 'postgres', for example, should
be an installation-wide username even when you're using db-local names
for ordinary users.

> This may require raising the length of NAME type to be backwards
> compatible.
>>
>> Right, but we're planning to do that anyway.

> Yes, but that requires a protocol change, which we don't want to do for
> 7.3.

What? We've been discussing raising NAMEDATALEN for months, and no
one's claimed that it qualifies as a protocol version change.

regards, tom lane

Marc G. Fournier writes:

> Next, unless Peter knows how to do it already, I've gotta learn to make
> configure more intelligent, so that for all intents, the "pieces" look
> like one package when building, not just when coding ...

It is possible, but it's not going to work.

There is a lot of interdependent and shared C code that needs to be put
somewhere. The build infrastructure is not ready to handle missing sub-
or superdirectories at all. Where is all the documentation going to go?
How are the installation instructions going to cope with the fact that no
one knows where everything is?

This whole things is a worthwhile idea, to some extent, but a lot more
planning needs to be done. In the meantime I humbly suggest you look for
a better package manager rather than letting it all out on us.

--
Peter Eisentraut peter_e(at)gmx(dot)net

Tom Lane writes:

> We've had several proposals in this thread for complicated extensions
> to the user naming mechanism. I think that's overdesigning the feature,
> because we have *no* examples of real-world need for such things except
> for Marc's situation. Let's keep it simple until we see real use cases
> that can drive the design of something fancy.

I don't buy this argument. The reason we have no examples is that people
are happily using the feature and don't have any reason to tell the world
about it.

--
Peter Eisentraut peter_e(at)gmx(dot)net

Lamar Owen writes:

> > allow specification of configuration files in a different directory

> I am going to review the previous thread and attempt to distill what can be
> done. I will then post a summary of what I found, with potential commentary.
> If a consensus is reached, I'd like to see the feature in 7.3.

The end result of the discussion was that no one could come up with a
bright idea to secure the configuration files without doing anything bogus
during installation.

Another issue, which becomes even more problematic if you factor in the
WAL file location discussion, is that if we drive the location of the data
from the configuration file instead of vice versa, we need to have initdb
smart enough to read those files.

Finally, I recall that a major reason to have these files in a separate
place is to be able to share them. But that won't work because those
files contain port numbers, data directory locations, etc. which can't be
shared. That needs a better plan than possibly "use command-line options
to override".

--
Peter Eisentraut peter_e(at)gmx(dot)net

Marc G. Fournier writes:

> I haven't touched libpq yet ... I'm talking with the libpqxx guys right
> now concerning getting the standalone libpqxx to work, and will be working
> on figuring out how to get the 'master configure' to make use of the
> standalone configure once that is fixed ... I want to get one module to
> work cleanly before looking at any others ...

I fail to understand how this mess is going to achieve anything. If you
like, you can assemble or split the modules into trees or tarballs after
you have them checked out, or even after you have downloaded and unpacked
them. But a source code repository is not a package manager.

Moreover, I would really like it if there was *some* discussion before we
are presented with done deals.

--
Peter Eisentraut peter_e(at)gmx(dot)net

On Thursday 01 August 2002 04:06 pm, Peter Eisentraut wrote:
> Another issue, which becomes even more problematic if you factor in the
> WAL file location discussion, is that if we drive the location of the data
> from the configuration file instead of vice versa, we need to have initdb
> smart enough to read those files.

Hmm, I hadn't thought about that -- but I like that idea. Not exclusive of
the existing way, either. But alongside it. More thought required.

> Finally, I recall that a major reason to have these files in a separate
> place is to be able to share them. But that won't work because those
> files contain port numbers, data directory locations, etc. which can't be
> shared. That needs a better plan than possibly "use command-line options
> to override".

No, the major reason was to allow the config files to live in a different area
than the data files without symlink kludges. The reasons why an admin might
want this are manifold. The reason I want it is to simplify multiple
postmasters in an RPM installation.

You can then blow away the whole PGDATA tree and start from scratch without
losing your config files.

You had an idea along these lines, and I was quite OK with the majority of it.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> >> (Actually, what I'd prefer it do is try first for username, and
> >> then username(at)databasename if plain username isn't found.)
>
> > Yes, that would be very easy to do _except_ for pg_hba.conf which does a
> > first-match for username. We could get into trouble there by trying two
> > versions of the same name. Comments?
>
> Hm. I think we'd have to switch around the order of stuff so that we
> look at the flat-file copy of pg_shadow first. Then we'd know which
> flavor of name we have, and we can proceed with the pg_hba match.
>
> The reason it's worth doing this is that 'postgres', for example, should
> be an installation-wide username even when you're using db-local names
> for ordinary users.

Yes, that's why my code had a special case for 'postgres' or whatever
super-user name it was installed with. I think it is cleaner to just
read the install username. Also, right now, pg_pwd only contains
usernames that have passwords, not all of them.

> > This may require raising the length of NAME type to be backwards
> > compatible.
> >>
> >> Right, but we're planning to do that anyway.
>
> > Yes, but that requires a protocol change, which we don't want to do for
> > 7.3.
>
> What? We've been discussing raising NAMEDATALEN for months, and no
> one's claimed that it qualifies as a protocol version change.

I thought they were talking about increasing the length of the user NAME
that comes of the wire. That is currently 32. I see now he was just
talking about NAMEDATALEN. Good thing we are prepending the database
name after receiving the name.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Peter Eisentraut wrote:
> Tom Lane writes:
>
> > We've had several proposals in this thread for complicated extensions
> > to the user naming mechanism. I think that's overdesigning the feature,
> > because we have *no* examples of real-world need for such things except
> > for Marc's situation. Let's keep it simple until we see real use cases
> > that can drive the design of something fancy.
>
> I don't buy this argument. The reason we have no examples is that people
> are happily using the feature and don't have any reason to tell the world
> about it.

Well, we are going to find out in 7.3 when the secondary password file
is no longer supported.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Bruce Momjian wrote:
> Functions Returning Sets - done?

The basic capability is done, but a number of supporting capabilities
remain. These are the ones I hope to have done for 7.3:

- PL/pgSQL table function support: not started, but I may get help with
this.
- anonymous composite types: patch submitted
- stand-alone composite types: proposal submitted
- implicit stand-alone composite types on CREATE FUNCTION: proposal
submitted
- Move show_all_settings() from contrib/tablefunc to the backend and
create a system view using the same method as Neil's pg_locks view.

Additional refinements (streaming vs tuplestore, rescan pushed from
planner to executor, etc) will be 7.4 items.

Additionally on my personal TODO for 7.3 are:
- modify contrib/dblink to take advantage of table function and new
composite type capabilities
- submit string manipulation functions discussed with Thomas a few
weeks ago --> replace(), to_hex(), extract_tok()

Joe

On Wed, Jul 31, 2002 at 02:08:33PM -0300, Marc G. Fournier wrote:
>
> Who cares? Those that need a C++ interface will know where to find it,
> and will report bugs that they have ... why should it be tested on every
> platform when we *might* only have those on the Linux platform using it?

Well, portability's actually a lot better than that OS-wise. The thing
to worry about is compilers. I've got some changes from Clinton James
waiting in the wings until libpqxx's status and development home are
settled. Those changes will make it compile on the latest version of
Visual C++ (and I'll take some changes out again once they're no longer
needed for that purpose), and most of it seems to work.

> What happens if/when libpqxx becomes the 'old, crufty interface' and
> something newer and shinier comes along? Where do we draw the line at
> what is in the distribution? For instance, why pgaccess vs a platform
> independent version of PgAdmin vs PHPPgAdmin? Hell, IMHO, PHPPgAdmin
> would most likely be more useful as more sites out there are running PHP
> then likely have TCL installed ... but someone that is using TCL/AolServer
> would definitely think otherwise ...

Looking at it that way, it seems to me that the proper approach is to
cut out all interfaces that don't talk to the backend themselves--e.g.
the ones that build on top of libpq, like libpq++ and libpqxx do.

Of course my humble but thoroughly biased opinion is that libpq++ be
marked "legacy."

> By branching out the fat, we make it *easier* for someone to take on
> development of it ... would libpqxx ever have been developed if Joergen
> could have just worked on libpq++ in the first place, without having to
> submit patches?

Yes. Now STOP BRUTALIZING MY NAME!

...

Phew. I feel better now. What was I saying?

Ah, yes. What you say pretty much describes how libpqxx came to be: get
a local copy of libpq++, try to fix it on a carte-blanche basis, find
nothing salvageable, write from scratch building on libpq++'s experience.

That said, I do support the idea of separately administered projects for
the reasons you give. Looking at specifics first, the problem I'm stuck
with for now is that I can't really work on the thing until this point is
decided. Well I could, but not until the doctor lets me get back to it.
Which requires, among other things, that it not give me headaches. :-)

For the more general case, there's the problem of release management: who's
going to be taking care of synchronizing releases? This may require some
new infrastructure, such as a mailing list dedicated to the process, or one
restricted to subproject maintainers. Or something.

Perhaps the unbundling of subprojects justifies that version bump to 8.0
after all...

Jeroen

(Juliet Echo Romeo Oscar Echo November. Jeroen. No G. Note intricate
order of vowels. Phonetic spelling in English would be Yeroon. Try to
roll the "r" a little.)

On Fri, 2 Aug 2002, jtv wrote:

> Looking at it that way, it seems to me that the proper approach is to
> cut out all interfaces that don't talk to the backend themselves--e.g.
> the ones that build on top of libpq, like libpq++ and libpqxx do.

This is what my opinion is ... what I'm setting up right now with CVS is
meant to be a middle ground ...

> Of course my humble but thoroughly biased opinion is that libpq++ be
> marked "legacy."

No doubt, but, if we didn't "push" one interface over another, then it
would be up to the end-users themselves to decide which one to install ...

> > By branching out the fat, we make it *easier* for someone to take on
> > development of it ... would libpqxx ever have been developed if Joergen
> > could have just worked on libpq++ in the first place, without having to
> > submit patches?
>
> Yes.

Okay, then let's word it another way ... if libpq++ *wasn't* in the
repository and part of the distribution, would you have a) started working
on it sooner? b) would you have made it public earlier? c) would ppl
have started to use it and stop'd using libpq++?

Basically, with libpq++ in the distribution, we are endorsing its use, so
if we didn't put libpqxx into the repository, would ppl switch from the
'endorsed' to the 'unendorsed' version?

By having libpq++ in the repository, we are adding weight to it that it
wouldn't have if it were outside of the repository, making it more
difficult for 'alternatives' to pop in ...

> For the more general case, there's the problem of release management: who's
> going to be taking care of synchronizing releases? This may require some
> new infrastructure, such as a mailing list dedicated to the process, or one
> restricted to subproject maintainers. Or something.

Well, for now, I'd say keep the status quo of just using -hackers ...

On Thu, Aug 01, 2002 at 09:07:46PM -0300, Marc G. Fournier wrote:
>
> > Of course my humble but thoroughly biased opinion is that libpq++ be
> > marked "legacy."
>
> No doubt, but, if we didn't "push" one interface over another, then it
> would be up to the end-users themselves to decide which one to install ...

In theory, yes. In this case, however, I see two arguments for making
the distinction anyway:

1. Some people won't want to go to the trouble of comparing available
interfaces, so they may default to libpq++ because it's what they found
first, or because they find mentions of it as the official C++ interface.
I think it would be a shame to have new users default to libpq++ "by
accident." I think many users would prefer to rely on the PostgreSQL
team's judgment--as they do by choosing Postgres in the first place.

2. I get the impression that libpq++ sort of got stuck before it was
completed. For the time being libpqxx appears to have better maintenance
prospects. Users will want to be aware of this before making a choice.

> Okay, then let's word it another way ... if libpq++ *wasn't* in the
> repository and part of the distribution, would you have a) started working
> on it sooner? b) would you have made it public earlier? c) would ppl
> have started to use it and stop'd using libpq++?

I'm not sure there's much point to going into a single example in detail,
but for completeness' sake I'll just answer these:

a) Yes.
b) No, because in my case I was encouraged by team members' endorsement of
first my suggestions for libpq++, and later a full replacement. Without
that, and without an active libpq++ maintainer around, libpqxx might never
have gotten off the ground.
c) I'd like to think so, yes--but exposure would have been harder.

> Basically, with libpq++ in the distribution, we are endorsing its use, so
> if we didn't put libpqxx into the repository, would ppl switch from the
> 'endorsed' to the 'unendorsed' version?
>
> By having libpq++ in the repository, we are adding weight to it that it
> wouldn't have if it were outside of the repository, making it more
> difficult for 'alternatives' to pop in ...

I definitely agree here. See above.

> > For the more general case, there's the problem of release management: who's
> > going to be taking care of synchronizing releases? This may require some
> > new infrastructure, such as a mailing list dedicated to the process, or one
> > restricted to subproject maintainers. Or something.

This reminds me of another potential complication: how would unbundling
affect the licensing situation? Mixing and matching components is good
in many ways, but it could complicate the situation for end-users--who
probably like to be able to rely on the team's judgment on this issue as
well.

I feel compelled at this point to admit that I prefer the GPL. This is
a personal preference, which I set aside because I wanted and expected
libpqxx to become the standard C++ interface. Had these interfaces not
been bundled, I would have had less incentive to conform to Postgres'
licensing conditions. I think having a different license would have
made everyone's life a little harder.

Jeroen

(and yes, I'm trying to repair my From: lines!)

On Tue, Jul 30, 2002 at 11:50:38PM -0400, Bruce Momjian wrote:
> ecpg and bison issues - solved?

Not solved yet. And it's just a matter of time until we run into it with
the main parser grammar file as well. Bison upstream is working on
removing all those short ints, but I have yet to receive a version that
compiles ecpg grammar correctly.

No idea, if this will be fixed in the next month.

Michael
--
Michael Meskes
Michael(at)Fam-Meskes(dot)De
Go SF 49ers! Go Rhein Fire!
Use Debian GNU/Linux! Use PostgreSQL!

Michael Meskes <meskes(at)postgresql(dot)org> writes:
> On Tue, Jul 30, 2002 at 11:50:38PM -0400, Bruce Momjian wrote:
>> ecpg and bison issues - solved?

> Not solved yet. And it's just a matter of time until we run into it with
> the main parser grammar file as well.

Yeah, I've been worrying about that too. Any idea how close we are to
trouble in the main grammar?

> Bison upstream is working on
> removing all those short ints, but I have yet to receive a version that
> compiles ecpg grammar correctly.

If no solution is forthcoming, we might have to adopt plan B: find
another parser-generator tool. Whilst googling for bison info I came
across "Why Bison is Becoming Extinct"
http://www.acm.org/crossroads/xrds7-5/bison.html
which is a tad amusing at least. Now, it's anyone's guess whether any
of the tools he suggests are actually ready for prime time; they might
have practical limits much worse than bison's. But I got awfully
frustrated yesterday trying (once again) to get bison to allow a
schema-qualified type name in the syntax <typename> <literal string>.
I'm just about ready to consider alternatives.

Plan C would be to devote some work to minimizing the number of states
in the main grammar (I assume it's number of states that's the problem).
I doubt anyone's ever tried, so there might be enough low-hanging fruit
to get ecpg off the hook for awhile.

regards, tom lane

On Sun, Aug 11, 2002 at 11:12:57AM -0400, Tom Lane wrote:
> > Not solved yet. And it's just a matter of time until we run into it with
> > the main parser grammar file as well.
>
> Yeah, I've been worrying about that too. Any idea how close we are to
> trouble in the main grammar?

No idea. The ecpg grammar in the main tree has about 530 rules, while my
actual version is at nearly 550. The main grammar should be at about
440. So there's some room left.

> Plan C would be to devote some work to minimizing the number of states
> in the main grammar (I assume it's number of states that's the problem).
> I doubt anyone's ever tried, so there might be enough low-hanging fruit
> to get ecpg off the hook for awhile.

Actually I already ate the really low-hanging fruit. :-)

I did spend some time to reduce the states, albeit surely not to the
extent possible, but still it will mean quite some work I'm afraid.

Michael

P.S.: No repsonse by bison upstream yet, but I think he's on vacation
this week.

--
Michael Meskes
Michael(at)Fam-Meskes(dot)De
Go SF 49ers! Go Rhein Fire!
Use Debian GNU/Linux! Use PostgreSQL!