Re: libpq API incompatibility between 7.4 and 8.0

Martin Pitt has detected that the libpq API has changed incompatibly
between 7.4 and 8.0. This has the effect, for example, that 7.4's psql
cannot run with 8.0's libpq. Example:

$ LD_LIBRARY_PATH=/home/peter/devel/pg80/pg-install/lib /home/peter/devel/pg74/pg-install/bin/psql --help
/home/peter/devel/pg74/pg-install/bin/psql: relocation error: /home/peter/devel/pg74/pg-install/bin/psql: undefined symbol:get_progname

I haven't looked further, but we have to fix this urgently, I think.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Martin Pitt has detected that the libpq API has changed incompatibly
> between 7.4 and 8.0. This has the effect, for example, that 7.4's psql
> cannot run with 8.0's libpq.

[ shrug... ] I don't think we've ever guaranteed that anyway. I will
resist putting get_progname back into libpq, because it should never
have been there, at least not with that name: it's an undocumented
infringement on application namespace.

regards, tom lane

Hi!

(sorry for the additional addresses; I'm not subscribed to -hackers,
so my mail will last a while until it arrives there).

Tom Lane [2005-02-02 11:07 -0500]:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > Martin Pitt has detected that the libpq API has changed incompatibly
> > between 7.4 and 8.0. This has the effect, for example, that 7.4's psql
> > cannot run with 8.0's libpq.
>
> [ shrug... ] I don't think we've ever guaranteed that anyway.

If you don't guarantee a backwards compatible API, then you should
give libpq a new SONAME. By keeping the SONAME "3" you did promise
backwards compatibility.

The problem with this issue is that it is a real pain for distributors
to handle broken SONAMEs. If every distributor invents his own one,
then there will be conflicts at some point. This gets worse if you do
release a new libpq SONAME later, which might conflict with any fake
SONAME a distributor might have invented.

> I will resist putting get_progname back into libpq, because it
> should never have been there, at least not with that name: it's an
> undocumented infringement on application namespace.

I understand such concerns, but you cannot undo the history. libpq3
is out there and installed on maybe hundreds of thousands of machines,
and your own psql frontend uses it.

The next best option would be to fix the 7.4 version of psql to not
use this symbol any more. However, this is still a pain since then you
cannot upgrade from earlier versions to 8.0 any more.

What would you propose as a solution? There can only be one
/usr/lib/libpq.so.3.

Thanks,

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian GNU/Linux Developer http://www.debian.org

Martin Pitt <mpitt(at)debian(dot)org> writes:
> What would you propose as a solution?

Do nothing. The problem you are raising isn't very serious since
RPM-style installations don't support concurrent installation of
multiple PG versions anyway. That being the case, it doesn't really
matter whether 8.0 psql can use a 7.4 library or vice versa.

To do otherwise would essentially amount to deciding that get_progname
is part of the exported API of libpq forevermore. That's not something
I'm willing to buy into. It was a mistake that it was done that way in
7.4, and I want to rectify that mistake before it gets any more
entrenched.

regards, tom lane

Tom Lane wrote:
> Martin Pitt <mpitt(at)debian(dot)org> writes:
> > What would you propose as a solution?
>
> Do nothing. The problem you are raising isn't very serious since
> RPM-style installations don't support concurrent installation of
> multiple PG versions anyway. That being the case, it doesn't really
> matter whether 8.0 psql can use a 7.4 library or vice versa.

Partial upgrades is the key word.

> To do otherwise would essentially amount to deciding that
> get_progname is part of the exported API of libpq forevermore.
> That's not something I'm willing to buy into. It was a mistake that
> it was done that way in 7.4, and I want to rectify that mistake
> before it gets any more entrenched.

We can rectify the mistake, but then we need to change the SONAME.
That's what it's for.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> We can rectify the mistake, but then we need to change the SONAME.
> That's what it's for.

Well, if you just want to bump libpq's SO_MAJOR_VERSION, I won't object.

This brings up a point that I think has been discussed before: we
operate on the assumption that incrementing SO_MINOR_VERSION is enough
to distinguish different releases of libpq, but in point of fact it
is not. The Linux conventions for library names, for one, essentially
require us to bump SO_MAJOR_VERSION for every release if we want to have
any hope of letting different versions coexist. Perhaps our convention
should be to bump SO_MAJOR_VERSION for each of our major releases and
only use SO_MINOR_VERSION when we change a library in a dot-release.

regards, tom lane

Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > We can rectify the mistake, but then we need to change the SONAME.
> > That's what it's for.
>
> Well, if you just want to bump libpq's SO_MAJOR_VERSION, I won't object.
>
> This brings up a point that I think has been discussed before: we
> operate on the assumption that incrementing SO_MINOR_VERSION is enough
> to distinguish different releases of libpq, but in point of fact it
> is not. The Linux conventions for library names, for one, essentially
> require us to bump SO_MAJOR_VERSION for every release if we want to have
> any hope of letting different versions coexist. Perhaps our convention
> should be to bump SO_MAJOR_VERSION for each of our major releases and
> only use SO_MINOR_VERSION when we change a library in a dot-release.

According to our RELEASE_CHANGES documentation:

The major version number should be updated whenever the source of the
library changes to make it binary incompatible. Such changes include,
but are not limited to:

1. Removing a public function or structure (or typedef, enum, ...)

2. Modifying a public functions arguments.

3. Removing a field from a public structure.

so while I don't think we need to update the major number for every
PostgreSQL major release, the removal of prog_name probably required a
major bump.

Oops.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> According to our RELEASE_CHANGES documentation:

> The major version number should be updated whenever the source of the
> library changes to make it binary incompatible. Such changes include,
> but are not limited to:

> 1. Removing a public function or structure (or typedef, enum, ...)

> 2. Modifying a public functions arguments.

> 3. Removing a field from a public structure.

> so while I don't think we need to update the major number for every
> PostgreSQL major release, the removal of prog_name probably required a
> major bump.

Well, the point is that get_progname *isn't* a "public" function.
We never advertised it as a libpq entry point.

What this really brings out to me is that our development process
doesn't impose a very strong boundary between libpq and our bundled
client programs. If the client programs were enforced to use only the
documented public API of libpq then we'd not be having this discussion
--- but stuff such as libpgport support functions tends to slip by under
the radar. IIRC we've been bitten in exactly this way at least once
before. What I'm suggesting is that we just solve the whole class of
problems permanently, by abandoning the assumption that we're going to
guarantee binary compatibility across major releases. I don't think
that promise is really buying us anything very critical.

If we don't go that way, then we need to have some automatic check that
none of the client programs are using symbols they shouldn't be from
libpq. (Hmm ... will the existence of the Windows port help here?)

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > According to our RELEASE_CHANGES documentation:
>
> > The major version number should be updated whenever the source of the
> > library changes to make it binary incompatible. Such changes include,
> > but are not limited to:
>
> > 1. Removing a public function or structure (or typedef, enum, ...)
>
> > 2. Modifying a public functions arguments.
>
> > 3. Removing a field from a public structure.
>
> > so while I don't think we need to update the major number for every
> > PostgreSQL major release, the removal of prog_name probably required a
> > major bump.
>
> Well, the point is that get_progname *isn't* a "public" function.
> We never advertised it as a libpq entry point.
>
> What this really brings out to me is that our development process
> doesn't impose a very strong boundary between libpq and our bundled
> client programs. If the client programs were enforced to use only the
> documented public API of libpq then we'd not be having this discussion
> --- but stuff such as libpgport support functions tends to slip by under
> the radar. IIRC we've been bitten in exactly this way at least once
> before. What I'm suggesting is that we just solve the whole class of
> problems permanently, by abandoning the assumption that we're going to
> guarantee binary compatibility across major releases. I don't think
> that promise is really buying us anything very critical.
>
> If we don't go that way, then we need to have some automatic check that
> none of the client programs are using symbols they shouldn't be from
> libpq. (Hmm ... will the existence of the Windows port help here?)

Yes, I think Win32 will help as long as we don't let bad stuff get into
libpqddll.def. The only downside I see to bumping the major number each
time is that the major number could get pretty big. Do the dynamic
library systems handle two-digit library version numbers properly?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> The only downside I see to bumping the major
> number each time is that the major number could get pretty big. Do
> the dynamic library systems handle two-digit library version numbers
> properly?

MySQL's client library is at 12, so I don't see a problem.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Hi Tom!

Tom Lane [2005-02-02 12:01 -0500]:
> Martin Pitt <mpitt(at)debian(dot)org> writes:
> > What would you propose as a solution?
>
> Do nothing.

That's unfortunately not an option.

> The problem you are raising isn't very serious since
> RPM-style installations don't support concurrent installation of
> multiple PG versions anyway. That being the case, it doesn't really
> matter whether 8.0 psql can use a 7.4 library or vice versa.

That's exactly the point I want to change for Debian, I work on a
structure which permits to run several clusters of different versions
in parallel (see [1]). We now have a /usr/lib/libpq.so.3 for 7.4, and
we can't have a _second_ incompatible /usr/lib/libpq.so.3 for 8.0.

Sorry, but that's just the way how shared libraries work. SONAMEs are
for API compatibility and it becomes _very_ hard to workaround broken
ones.

> To do otherwise would essentially amount to deciding that get_progname
> is part of the exported API of libpq forevermore.

That's not true. It must stay part of the exported API for SONAME 3 of
libpq, not for anything else.

> That's not something I'm willing to buy into. It was a mistake that
> it was done that way in 7.4, and I want to rectify that mistake
> before it gets any more entrenched.

Then please release 8.0.2 with a SONAME 4. Would you consider this?

Thanks a lot and have a nice day!

Martin

[1] http://people.debian.org/~mpitt/postgresql-ng.html

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian GNU/Linux Developer http://www.debian.org

Tom Lane wrote:
> Well, if you just want to bump libpq's SO_MAJOR_VERSION, I won't
> object.

Yes. Unless someone objects, I will do that for 8.0.* and 8.1.*.

> The Linux conventions for library names, for one,
> essentially require us to bump SO_MAJOR_VERSION for every release if
> we want to have any hope of letting different versions coexist.
> Perhaps our convention should be to bump SO_MAJOR_VERSION for each of
> our major releases and only use SO_MINOR_VERSION when we change a
> library in a dot-release.

Well, the goal for coexistence is mainly the server and the
server-dependent tools (pg_dump, psql, etc.). There is no per-se
requirement to have libpq versions to coexist. If libpq has different
sonames in different releases, then they would end up in differently
named packages anyway (at least in Debian). If they don't have
different sonames, then we declare that they are compatible, so it
should be OK to have only the latest version installed. That requires
us to stay honest with the sonames, but it does not require us to
increase the sonames unnecessarily.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Tom Lane wrote:
> > Well, if you just want to bump libpq's SO_MAJOR_VERSION, I won't
> > object.
>
> Yes. Unless someone objects, I will do that for 8.0.* and 8.1.*.

I am thinking we should up the 8.0.* and 8.1.* releases to have the same
major number, but not make a major libpq bump for every major release
unless it is required, like with prog_name. Is that the plan?

I guess the big question is whether we would ever want a 7.2 psql or any
other client to automatically use a 7.3 libpq. In fact by upping the
major every time will 7.2 clients automatically use the 7.3 libpq or
will they have to be relinked?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> ... If they don't have
> different sonames, then we declare that they are compatible, so it
> should be OK to have only the latest version installed. That requires
> us to stay honest with the sonames, but it does not require us to
> increase the sonames unnecessarily.

Well, the problem I'm concerned about is how do we stay honest. We now
realize that our devel procedures may not catch this sort of mistake.

It might be that the Windows DLLs will catch any unintentional
dependencies, but I'm not feeling especially comfortable about it.

regards, tom lane

Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > ... If they don't have
> > different sonames, then we declare that they are compatible, so it
> > should be OK to have only the latest version installed. That requires
> > us to stay honest with the sonames, but it does not require us to
> > increase the sonames unnecessarily.
>
> Well, the problem I'm concerned about is how do we stay honest. We now
> realize that our devel procedures may not catch this sort of mistake.
>
> It might be that the Windows DLLs will catch any unintentional
> dependencies, but I'm not feeling especially comfortable about it.

I am hopefull Win32 will help here and am willing to give it another
try.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> Peter Eisentraut wrote:
> > Tom Lane wrote:
> > > Well, if you just want to bump libpq's SO_MAJOR_VERSION, I won't
> > > object.
> >
> > Yes. Unless someone objects, I will do that for 8.0.* and 8.1.*.
>
> I am thinking we should up the 8.0.* and 8.1.* releases to have the same
> major number, but not make a major libpq bump for every major release
> unless it is required, like with prog_name. Is that the plan?
>
> I guess the big question is whether we would ever want a 7.2 psql or any
> other client to automatically use a 7.3 libpq. In fact by upping the
> major every time will 7.2 clients automatically use the 7.3 libpq or
> will they have to be relinked?

Uh, if we bump up the major library version in 8.0.X, will that require
8.0.0 user applications to be recompiled? That seems worse than having
7.4.X pg apps like psql fail. They fail because they are calling
get_progname, but most user apps will not be calling that function.

Also, if we bump the major version, will an install of 8.0.X fix
problems with get_progname? Won't the 8.0.0 libpq with a higher minor
version number still prevent old psql's from running? Our install
doesn't uninstall old libraries, does it? RPM's might but I am not sure
how they do minor upgrades.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:

> In fact by upping the major every time will 7.2 clients automatically use
> the 7.3 libpq or will they have to be relinked?

If you do not bump the soname then 7.2 clients will automatically immediately
start using the new library when it's installed. (actually when ldconfig is
run, which distributions will normally do automatically).

If you do bump the versions then 7.2 clients will continue to use 7.2
libraries. If you have 7.2 clients, or even clients like DBD::Pg or other
third-party libpq apps then you would have to relink them to use the new
libraries. But you would probably just keep around both sets of libraries.

I could see valid arguments for either when it comes to whether to bump it
when it's not strictly necessary.

If libpq were used by lots of external applications using the public interface
then bumping it more than necessary means people have to keep around extra
versions for no reason.

On the other hand if libpq is seen as an integral part of the postgres package
and its specific behaviour is important then not bumping the version forces
admins to pick one version for everything when they might prefer that 7.2
clients stick with the 7.2 library.

In a way the fact that you release new libpq versions when you release
security releases for old releases like 7.3 or 7.4 sort of already implies an
answer. If you thought the new library was plug-in compatible and should
completely replace the old library automatically you wouldn't continue to
release modified versions of the old library, you would just release HEAD for
libpq all the time.

--
greg

Bruce Momjian wrote:
> Uh, if we bump up the major library version in 8.0.X, will that
> require 8.0.0 user applications to be recompiled?

No, they just keep using the old library.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > Uh, if we bump up the major library version in 8.0.X, will that
> > require 8.0.0 user applications to be recompiled?
>
> No, they just keep using the old library.

That assumes the old libraries stay around. Will they?

I am thinking the easiest solution will be to re-add get_progname() to
8.0.X and bump the major for 8.1.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Hi!

Bruce Momjian [2005-02-03 9:42 -0500]:
> Peter Eisentraut wrote:
> > Bruce Momjian wrote:
> > > Uh, if we bump up the major library version in 8.0.X, will that
> > > require 8.0.0 user applications to be recompiled?
> >
> > No, they just keep using the old library.
>
> That assumes the old libraries stay around. Will they?
>
> I am thinking the easiest solution will be to re-add get_progname() to
> 8.0.X and bump the major for 8.1.

Seconded. Then we don't need another library version, and it is still
not necessary to drag this get_progname accident forever.

Thanks,

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian GNU/Linux Developer http://www.debian.org

Martin Pitt <mpitt(at)debian(dot)org> writes:
>> I am thinking the easiest solution will be to re-add get_progname() to
>> 8.0.X and bump the major for 8.1.

> Seconded. Then we don't need another library version, and it is still
> not necessary to drag this get_progname accident forever.

We're going to stop dragging the accident now, not a year from now.

Putting get_progname back into libpq would require re-adding all of
path.c, which creates many more issues than just adjusting
SO_MAJOR_VERSION would do.

regards, tom lane

Tom Lane wrote:

>Martin Pitt <mpitt(at)debian(dot)org> writes:
>
>
>>>I am thinking the easiest solution will be to re-add get_progname() to
>>>8.0.X and bump the major for 8.1.
>>>
>>>
>
>
>
>>Seconded. Then we don't need another library version, and it is still
>>not necessary to drag this get_progname accident forever.
>>
>>
>
>We're going to stop dragging the accident now, not a year from now.
>
>Putting get_progname back into libpq would require re-adding all of
>path.c, which creates many more issues than just adjusting
>SO_MAJOR_VERSION would do.
>
>
>
>

Maybe I'm dense, but I don't understand why this is causing anyone a
headache. Why would one install the 8.0 libs without the 8.0 clients?

cheers

andrew

Hi!

Andrew Dunstan [2005-02-03 11:24 -0500]:
> Maybe I'm dense, but I don't understand why this is causing anyone a
> headache. Why would one install the 8.0 libs without the 8.0 clients?

That's not the point. The point is that this breakage makes it
impossible to install _both_ 7.4 and 8.0 server/client at the same
time.

In addition, this breaks partial upgrades: if the package libpq3 (as
it is called in Debian) is upgraded from 7.4.x to 8.0.x (which is
perfectly valid and does not break any dependencies, since it is the
same library with the same API), but the postgresql-client package
isn't (because it is on hold, or whatever), then this breaks the old
postgresql-client as well.

I'm fine with immediately bumping the SONAME too, but as I said, it's
important that this happens upstream, not in the distributions.

Thanks,

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian GNU/Linux Developer http://www.debian.org

Tom Lane wrote:
> Martin Pitt <mpitt(at)debian(dot)org> writes:
> >> I am thinking the easiest solution will be to re-add get_progname() to
> >> 8.0.X and bump the major for 8.1.
>
> > Seconded. Then we don't need another library version, and it is still
> > not necessary to drag this get_progname accident forever.
>
> We're going to stop dragging the accident now, not a year from now.
>
> Putting get_progname back into libpq would require re-adding all of
> path.c, which creates many more issues than just adjusting
> SO_MAJOR_VERSION would do.

Why can't we just put a copy of get_progname into libpq, rather than
bring the entire file from libpgport?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Martin Pitt wrote:
-- Start of PGP signed section.
> Hi!
>
> Andrew Dunstan [2005-02-03 11:24 -0500]:
> > Maybe I'm dense, but I don't understand why this is causing anyone a
> > headache. Why would one install the 8.0 libs without the 8.0 clients?
>
> That's not the point. The point is that this breakage makes it
> impossible to install _both_ 7.4 and 8.0 server/client at the same
> time.
>
> In addition, this breaks partial upgrades: if the package libpq3 (as
> it is called in Debian) is upgraded from 7.4.x to 8.0.x (which is
> perfectly valid and does not break any dependencies, since it is the
> same library with the same API), but the postgresql-client package
> isn't (because it is on hold, or whatever), then this breaks the old
> postgresql-client as well.
>
> I'm fine with immediately bumping the SONAME too, but as I said, it's
> important that this happens upstream, not in the distributions.

Are users going to be happy to have to recompile all their user apps
that use libpq because of a minor upgrade? We have required it in the
past, 6.3.2:

A dump/restore is NOT required for those running 6.3 or 6.3.1. A make
distclean, make, and make install is all that is required. This last
step should be performed while the postmaster is not running. You should
re-link any custom applications that use PostgreSQL libraries.

but in this case their apps are running fine and it is just old pg
clients that need the fix.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Am Donnerstag, 3. Februar 2005 15:42 schrieb Bruce Momjian:
> Peter Eisentraut wrote:
> > Bruce Momjian wrote:
> > > Uh, if we bump up the major library version in 8.0.X, will that
> > > require 8.0.0 user applications to be recompiled?
> >
> > No, they just keep using the old library.
>
> That assumes the old libraries stay around. Will they?

Well, if someone did remove them then they didn't understand the point of the
whole thing and should not be allowed to install libraries without
supervision. :)

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Am Donnerstag, 3. Februar 2005 15:42 schrieb Bruce Momjian:
> > Peter Eisentraut wrote:
> > > Bruce Momjian wrote:
> > > > Uh, if we bump up the major library version in 8.0.X, will that
> > > > require 8.0.0 user applications to be recompiled?
> > >
> > > No, they just keep using the old library.
> >
> > That assumes the old libraries stay around. Will they?
>
> Well, if someone did remove them then they didn't understand the point of the
> whole thing and should not be allowed to install libraries without
> supervision. :)

I was asking if the 8.0.0 libpq stays around. If it does then the 7.4.X
libpq will still see the 8.0.0 libpq and will still not work.

That's why the get_progname() addition would be cleaner in some ways.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> I was asking if the 8.0.0 libpq stays around. If it does then the 7.4.X
> libpq will still see the 8.0.0 libpq and will still not work.

> That's why the get_progname() addition would be cleaner in some ways.

How you figure that? Your first conclusion assumes that someone updates
an 8.0.0 installation and fails to replace the 8.0.0 libpq, while your
second conclusion assumes that they do replace the 8.0.0 libpq. This is
unlikely in any package-based distribution (RPM doesn't forget such things)
and if they built from source they have many other ways besides this to
shoot themselves in the foot (like configuring SSL support one time and
not the next).

This problem isn't worth spending more development time on than it takes
to change SO_MAJOR_VERSION (we have lots of higher-priority issues).
And it definitely isn't worth exposing the path.c symbols for a second
release cycle and thereby doubling the odds that some outside code comes
to depend on them ... in which case we'd *really* have a problem.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > I was asking if the 8.0.0 libpq stays around. If it does then the 7.4.X
> > libpq will still see the 8.0.0 libpq and will still not work.
>
> > That's why the get_progname() addition would be cleaner in some ways.
>
> How you figure that? Your first conclusion assumes that someone updates
> an 8.0.0 installation and fails to replace the 8.0.0 libpq, while your
> second conclusion assumes that they do replace the 8.0.0 libpq. This is
> unlikely in any package-based distribution (RPM doesn't forget such things)
> and if they built from source they have many other ways besides this to
> shoot themselves in the foot (like configuring SSL support one time and
> not the next).

My point is that some will replace the 8.0.0 libpq (RPM), while others
will not (source install), and that will lead to different failure
cases.

The first will lead to the requirement that all user applications be
recompiled, and the later will lead to 7.4.X psql still not working.

> This problem isn't worth spending more development time on than it takes
> to change SO_MAJOR_VERSION (we have lots of higher-priority issues).

Those failure cases are worth addressing.

> And it definitely isn't worth exposing the path.c symbols for a second
> release cycle and thereby doubling the odds that some outside code comes
> to depend on them ... in which case we'd *really* have a problem.

I suggested to just get_progname() to libpq, not all of path.c. The
odds someone will depend on get_progname() in 8.0 are much less than the
problems we will have as listed above.

I like symbol cleanliness as much as the rest of you but I don't see a
need to cause user problems to fix it in 8.0.X.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Am Freitag, 4. Februar 2005 17:51 schrieb Bruce Momjian:
> I suggested to just get_progname() to libpq, not all of path.c. The
> odds someone will depend on get_progname() in 8.0 are much less than the
> problems we will have as listed above.

Perhaps a question is in order: Are we sure that get_progname() is the only
problem, or just the first one the linker found? Does anyone remember?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Am Freitag, 4. Februar 2005 17:51 schrieb Bruce Momjian:
>> I suggested to just get_progname() to libpq, not all of path.c. The
>> odds someone will depend on get_progname() in 8.0 are much less than the
>> problems we will have as listed above.

> Perhaps a question is in order: Are we sure that get_progname() is the only
> problem, or just the first one the linker found? Does anyone remember?

The report only showed that that one was the first the linker found.

The 7.4 libpq does include path.c in its entirety, meaning that there
are potentially dependencies out there for all of the symbols defined by
7.4 path.c: is_absolute_path, first_path_separator, last_path_separator,
in addition to get_progname. In the 8.0 version of path.c these symbols
are intertwined with a bunch more; what's more, is_absolute_path is now
a macro not a routine, so we lost ABI compatibility anyway. (I do in
fact see is_absolute_path used in 7.4 psql; didn't bother looking for
the others.)

In short, fixing this the way Bruce wants to will be a nontrivial amount
of effort.

regards, tom lane

Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > Am Freitag, 4. Februar 2005 17:51 schrieb Bruce Momjian:
> >> I suggested to just get_progname() to libpq, not all of path.c. The
> >> odds someone will depend on get_progname() in 8.0 are much less than the
> >> problems we will have as listed above.
>
> > Perhaps a question is in order: Are we sure that get_progname() is the only
> > problem, or just the first one the linker found? Does anyone remember?
>
> The report only showed that that one was the first the linker found.
>
> The 7.4 libpq does include path.c in its entirety, meaning that there
> are potentially dependencies out there for all of the symbols defined by
> 7.4 path.c: is_absolute_path, first_path_separator, last_path_separator,
> in addition to get_progname. In the 8.0 version of path.c these symbols
> are intertwined with a bunch more; what's more, is_absolute_path is now
> a macro not a routine, so we lost ABI compatibility anyway. (I do in
> fact see is_absolute_path used in 7.4 psql; didn't bother looking for
> the others.)
>
> In short, fixing this the way Bruce wants to will be a nontrivial amount
> of effort.

psql actually calls get_progname(). Do we know that it will try to link
in the other functions from path.c? I am unsure.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
>> In short, fixing this the way Bruce wants to will be a nontrivial amount
>> of effort.

> psql actually calls get_progname(). Do we know that it will try to link
> in the other functions from path.c? I am unsure.

I don't know of any commonly used linkers that link at granularity finer
than one source file, and in any case for shared libraries the library
is all-or-nothing.

Grep shows that 7.4 psql does reference is_absolute_path, and 7.4
pg_dumpall does reference last_path_separator, so at least three of the
four are at issue.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> >> In short, fixing this the way Bruce wants to will be a nontrivial amount
> >> of effort.
>
> > psql actually calls get_progname(). Do we know that it will try to link
> > in the other functions from path.c? I am unsure.
>
> I don't know of any commonly used linkers that link at granularity finer
> than one source file, and in any case for shared libraries the library
> is all-or-nothing.
>
> Grep shows that 7.4 psql does reference is_absolute_path, and 7.4
> pg_dumpall does reference last_path_separator, so at least three of the
> four are at issue.

Ewe, yep, that is a mess.

OK, so how will we handle this? We will bump the major for 8.0.X.

Do we also hack libpq/Makefile to remove the 8.0.0 libpq? If not, then
RPM folks will need to relink, but source installs will not, but they
will have to remove the 8.0.0 libpq if they also want to run 7.4.

As you can see it is the confusion that bothers me. I am not sure how I
would write a coherent paragraph explaining this.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> As you can see it is the confusion that bothers me. I am not sure how I
> would write a coherent paragraph explaining this.

The same thing you wrote the last time we had to do this (7.3.1).
I don't recall any huge volume of complaints last time, so I think
you're making a mountain out of a molehill.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > As you can see it is the confusion that bothers me. I am not sure how I
> > would write a coherent paragraph explaining this.
>
> The same thing you wrote the last time we had to do this (7.3.1).
> I don't recall any huge volume of complaints last time, so I think
> you're making a mountain out of a molehill.

OK, agreed.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Hi!

Tom Lane [2005-02-03 11:12 -0500]:
> Martin Pitt <mpitt(at)debian(dot)org> writes:
> >> I am thinking the easiest solution will be to re-add get_progname() to
> >> 8.0.X and bump the major for 8.1.
>
> > Seconded. Then we don't need another library version, and it is still
> > not necessary to drag this get_progname accident forever.
>
> We're going to stop dragging the accident now, not a year from now.
>
> Putting get_progname back into libpq would require re-adding all of
> path.c, which creates many more issues than just adjusting
> SO_MAJOR_VERSION would do.

The consensus seems to be to just bump the SONAME. The next one will
definitively be "4" then? I would like to fix this in my packages
before I announce them for public testing, but this issue is the only
blocker to it.

Will there be a new bugfix release soon?

Thanks for your work and have a nice day!

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian GNU/Linux Developer http://www.debian.org

Hi!

Tom Lane [2005-02-04 10:27 -0500]:
> This problem isn't worth spending more development time on than it takes
> to change SO_MAJOR_VERSION (we have lots of higher-priority issues).

I just did that:

--- postgresql-8.0.1-old/src/interfaces/libpq/Makefile 2005-01-26 20:24:19.000000000 +0100
+++ postgresql-8.0.1/src/interfaces/libpq/Makefile 2005-02-07 20:09:38.719686432 +0100
@@ -16,8 +16,8 @@

# shared library parameters
NAME= pq
-SO_MAJOR_VERSION= 3
-SO_MINOR_VERSION= 2
+SO_MAJOR_VERSION= 4
+SO_MINOR_VERSION= 0
DLTYPE= library

override CPPFLAGS := -DFRONTEND -I$(srcdir) $(CPPFLAGS) -I$(top_builddir)/src/port

Is that right? Or will you use SO_MINOR_VERSION=1?

However, I just stumbled across another problem: libpq3 and the new
libpq4 use the same translation domain "libpq4", thus they cannot be
installed in parallel. Can you please change the domain to "libpq4" as
well? This should generally be done anyway, but at least we can live
with one breakage of this rule (for libpq3), so that we don't need to
mess with the old libpq3 package.

Thanks,

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian GNU/Linux Developer http://www.debian.org

Martin Pitt wrote:
-- Start of PGP signed section.
> Hi!
>
> Tom Lane [2005-02-04 10:27 -0500]:
> > This problem isn't worth spending more development time on than it takes
> > to change SO_MAJOR_VERSION (we have lots of higher-priority issues).
>
> I just did that:
>
> --- postgresql-8.0.1-old/src/interfaces/libpq/Makefile 2005-01-26 20:24:19.000000000 +0100
> +++ postgresql-8.0.1/src/interfaces/libpq/Makefile 2005-02-07 20:09:38.719686432 +0100
> @@ -16,8 +16,8 @@
>
> # shared library parameters
> NAME= pq
> -SO_MAJOR_VERSION= 3
> -SO_MINOR_VERSION= 2
> +SO_MAJOR_VERSION= 4
> +SO_MINOR_VERSION= 0
> DLTYPE= library
>
> override CPPFLAGS := -DFRONTEND -I$(srcdir) $(CPPFLAGS) -I$(top_builddir)/src/port
>
> Is that right? Or will you use SO_MINOR_VERSION=1?

It would be zero. I wouldn't distribute this until we actually release
it, but for testing I guess it is OK.

> However, I just stumbled across another problem: libpq3 and the new
> libpq4 use the same translation domain "libpq4", thus they cannot be
> installed in parallel. Can you please change the domain to "libpq4" as
> well? This should generally be done anyway, but at least we can live
> with one breakage of this rule (for libpq3), so that we don't need to
> mess with the old libpq3 package.

Uh, what is the translation domain?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Hi!

Bruce Momjian [2005-02-09 18:05 -0500]:
> > However, I just stumbled across another problem: libpq3 and the new
> > libpq4 use the same translation domain "libpq4", thus they cannot be
> > installed in parallel. Can you please change the domain to "libpq4" as
> > well? This should generally be done anyway, but at least we can live
> > with one breakage of this rule (for libpq3), so that we don't need to
> > mess with the old libpq3 package.
>
> Uh, what is the translation domain?

In short, that is the unique name for a bunch of translations for a
particular application or group of related applications. The key point
is that the name of the file in
/usr/share/locale/<LOCALE>/LC_MESSAGES/<DOMAIN>.mo and the call

bindtextdomain ("<DOMAIN>", NULL)

must use the same <DOMAIN> so that libintl can find the mo file.
However, that means that all applications that are installed in
parallel need a distinct domain. Since the whole point of SONAMes is
to allow several different library API versions to be installed in
parallel, every library API (i. e. SONAME) should have an unique
domain, which can be achieved easiest by just appending the SONAME to
the translation domain.

In my test packages I used the following patch:

diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c postgresql-8.0.1/src/interfaces/libpq/fe-misc.c
--- postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c 2004-12-31 23:03:50.000000000 +0100
+++ postgresql-8.0.1/src/interfaces/libpq/fe-misc.c 2005-02-07 22:55:13.177419296 +0100
@@ -1133,7 +1133,7 @@
{
already_bound = 1;
/* No relocatable lookup here because the binary could be anywhere */
- bindtextdomain("libpq", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
+ bindtextdomain("libpq4", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
}

return dgettext("libpq", msgid);
diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/nls.mk postgresql-8.0.1/src/interfaces/libpq/nls.mk
--- postgresql-8.0.1-old/src/interfaces/libpq/nls.mk 2005-01-14 09:57:06.000000000 +0100
+++ postgresql-8.0.1/src/interfaces/libpq/nls.mk 2005-02-07 22:54:54.770217616 +0100
@@ -1,5 +1,5 @@
# $PostgreSQL: pgsql/src/interfaces/libpq/nls.mk,v 1.20 2005/01/14 08:57:06 petere Exp $
-CATALOG_NAME := libpq
+CATALOG_NAME := libpq4
AVAIL_LANGUAGES := af cs de es fr hr it ko nb pl pt_BR ru sk sl sv tr zh_CN zh_TW
GETTEXT_FILES := fe-auth.c fe-connect.c fe-exec.c fe-lobj.c fe-misc.c fe-protocol2.c fe-protocol3.c fe-secure.c
GETTEXT_TRIGGERS:= libpq_gettext pqInternalNotice:2

Compared to the SONAME, changing the translation domain is relatively
uncritical, so if you don't want to change this upstream, I can
maintain this patch for Debian/Ubuntu. However, I heard that some RPM
guys plan a infrastructure similar to mine, and at that point they
will have precisely the same problems :-)

Thanks for considering and have a nice day!

Martin
--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian GNU/Linux Developer http://www.debian.org

Martin Pitt wrote:
-- Start of PGP signed section.
> Hi!
>
> Bruce Momjian [2005-02-09 18:05 -0500]:
> > > However, I just stumbled across another problem: libpq3 and the new
> > > libpq4 use the same translation domain "libpq4", thus they cannot be
> > > installed in parallel. Can you please change the domain to "libpq4" as
> > > well? This should generally be done anyway, but at least we can live
> > > with one breakage of this rule (for libpq3), so that we don't need to
> > > mess with the old libpq3 package.
> >
> > Uh, what is the translation domain?
>
> In short, that is the unique name for a bunch of translations for a
> particular application or group of related applications. The key point
> is that the name of the file in
> /usr/share/locale/<LOCALE>/LC_MESSAGES/<DOMAIN>.mo and the call
>
> bindtextdomain ("<DOMAIN>", NULL)

Well, that is certainly interesting.

> must use the same <DOMAIN> so that libintl can find the mo file.
> However, that means that all applications that are installed in
> parallel need a distinct domain. Since the whole point of SONAMes is
> to allow several different library API versions to be installed in
> parallel, every library API (i. e. SONAME) should have an unique
> domain, which can be achieved easiest by just appending the SONAME to
> the translation domain.

I see. So we have to auto-populate the SONAME into all those places.
Now that you mention it, it makes sense because the language strings are
fixed for every major release so I can see the need for this when using
two major versions of PostgreSQL.

---------------------------------------------------------------------------

>
> In my test packages I used the following patch:
>
> diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c postgresql-8.0.1/src/interfaces/libpq/fe-misc.c
> --- postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c 2004-12-31 23:03:50.000000000 +0100
> +++ postgresql-8.0.1/src/interfaces/libpq/fe-misc.c 2005-02-07 22:55:13.177419296 +0100
> @@ -1133,7 +1133,7 @@
> {
> already_bound = 1;
> /* No relocatable lookup here because the binary could be anywhere */
> - bindtextdomain("libpq", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
> + bindtextdomain("libpq4", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
> }
>
> return dgettext("libpq", msgid);
> diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/nls.mk postgresql-8.0.1/src/interfaces/libpq/nls.mk
> --- postgresql-8.0.1-old/src/interfaces/libpq/nls.mk 2005-01-14 09:57:06.000000000 +0100
> +++ postgresql-8.0.1/src/interfaces/libpq/nls.mk 2005-02-07 22:54:54.770217616 +0100
> @@ -1,5 +1,5 @@
> # $PostgreSQL: pgsql/src/interfaces/libpq/nls.mk,v 1.20 2005/01/14 08:57:06 petere Exp $
> -CATALOG_NAME := libpq
> +CATALOG_NAME := libpq4
> AVAIL_LANGUAGES := af cs de es fr hr it ko nb pl pt_BR ru sk sl sv tr zh_CN zh_TW
> GETTEXT_FILES := fe-auth.c fe-connect.c fe-exec.c fe-lobj.c fe-misc.c fe-protocol2.c fe-protocol3.c fe-secure.c
> GETTEXT_TRIGGERS:= libpq_gettext pqInternalNotice:2
>
> Compared to the SONAME, changing the translation domain is relatively
> uncritical, so if you don't want to change this upstream, I can
> maintain this patch for Debian/Ubuntu. However, I heard that some RPM
> guys plan a infrastructure similar to mine, and at that point they
> will have precisely the same problems :-)
>
> Thanks for considering and have a nice day!
>
> Martin
> --
> Martin Pitt http://www.piware.de
> Ubuntu Developer http://www.ubuntulinux.org
> Debian GNU/Linux Developer http://www.debian.org
-- End of PGP section, PGP failed!

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Is this a direction we want to explore --- using the SONAME as part of
the translation domain?

---------------------------------------------------------------------------

Martin Pitt wrote:
-- Start of PGP signed section.
> Hi!
>
> Bruce Momjian [2005-02-09 18:05 -0500]:
> > > However, I just stumbled across another problem: libpq3 and the new
> > > libpq4 use the same translation domain "libpq4", thus they cannot be
> > > installed in parallel. Can you please change the domain to "libpq4" as
> > > well? This should generally be done anyway, but at least we can live
> > > with one breakage of this rule (for libpq3), so that we don't need to
> > > mess with the old libpq3 package.
> >
> > Uh, what is the translation domain?
>
> In short, that is the unique name for a bunch of translations for a
> particular application or group of related applications. The key point
> is that the name of the file in
> /usr/share/locale/<LOCALE>/LC_MESSAGES/<DOMAIN>.mo and the call
>
> bindtextdomain ("<DOMAIN>", NULL)
>
> must use the same <DOMAIN> so that libintl can find the mo file.
> However, that means that all applications that are installed in
> parallel need a distinct domain. Since the whole point of SONAMes is
> to allow several different library API versions to be installed in
> parallel, every library API (i. e. SONAME) should have an unique
> domain, which can be achieved easiest by just appending the SONAME to
> the translation domain.
>
> In my test packages I used the following patch:
>
> diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c postgresql-8.0.1/src/interfaces/libpq/fe-misc.c
> --- postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c 2004-12-31 23:03:50.000000000 +0100
> +++ postgresql-8.0.1/src/interfaces/libpq/fe-misc.c 2005-02-07 22:55:13.177419296 +0100
> @@ -1133,7 +1133,7 @@
> {
> already_bound = 1;
> /* No relocatable lookup here because the binary could be anywhere */
> - bindtextdomain("libpq", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
> + bindtextdomain("libpq4", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
> }
>
> return dgettext("libpq", msgid);
> diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/nls.mk postgresql-8.0.1/src/interfaces/libpq/nls.mk
> --- postgresql-8.0.1-old/src/interfaces/libpq/nls.mk 2005-01-14 09:57:06.000000000 +0100
> +++ postgresql-8.0.1/src/interfaces/libpq/nls.mk 2005-02-07 22:54:54.770217616 +0100
> @@ -1,5 +1,5 @@
> # $PostgreSQL: pgsql/src/interfaces/libpq/nls.mk,v 1.20 2005/01/14 08:57:06 petere Exp $
> -CATALOG_NAME := libpq
> +CATALOG_NAME := libpq4
> AVAIL_LANGUAGES := af cs de es fr hr it ko nb pl pt_BR ru sk sl sv tr zh_CN zh_TW
> GETTEXT_FILES := fe-auth.c fe-connect.c fe-exec.c fe-lobj.c fe-misc.c fe-protocol2.c fe-protocol3.c fe-secure.c
> GETTEXT_TRIGGERS:= libpq_gettext pqInternalNotice:2
>
> Compared to the SONAME, changing the translation domain is relatively
> uncritical, so if you don't want to change this upstream, I can
> maintain this patch for Debian/Ubuntu. However, I heard that some RPM
> guys plan a infrastructure similar to mine, and at that point they
> will have precisely the same problems :-)
>
> Thanks for considering and have a nice day!
>
> Martin
> --
> Martin Pitt http://www.piware.de
> Ubuntu Developer http://www.ubuntulinux.org
> Debian GNU/Linux Developer http://www.debian.org
-- End of PGP section, PGP failed!

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Mon, Jun 06, 2005 at 09:23:21PM -0400, Bruce Momjian wrote:
>
> Is this a direction we want to explore --- using the SONAME as part of
> the translation domain?

Hm, interesting -- this could explain some weird problems I've had with
translated text on a machine where multiple versions are installed.
I'll investigate a bit and if it turns out to be solved by this
approach, it has my vote.

However I think we should stay away from using the version number as a
string inside the source. If we do tht, we will forget to increment it
in the future and we will have problems.

> > In my test packages I used the following patch:
> >
> > diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c postgresql-8.0.1/src/interfaces/libpq/fe-misc.c
> > --- postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c 2004-12-31 23:03:50.000000000 +0100
> > +++ postgresql-8.0.1/src/interfaces/libpq/fe-misc.c 2005-02-07 22:55:13.177419296 +0100
> > @@ -1133,7 +1133,7 @@
> > {
> > already_bound = 1;
> > /* No relocatable lookup here because the binary could be anywhere */
> > - bindtextdomain("libpq", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
> > + bindtextdomain("libpq4", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
> > }

--
Alvaro Herrera (<alvherre[a]surnet.cl>)
"La experiencia nos dice que el hombre peló millones de veces las patatas,
pero era forzoso admitir la posibilidad de que en un caso entre millones,
las patatas pelarían al hombre" (Ijon Tichy)

Hi!

Bruce Momjian [2005-06-06 21:23 -0400]:
>
> Is this a direction we want to explore --- using the SONAME as part of
> the translation domain?

If that would go upstream, so much the better. I already do it in the
Debian and Ubuntu packages since I don't have any choice anyway, and
it's not really hard to maintain. But if it would go upstream, other
vendors might benefit from that as well.

However, as Alvaro already said, the patch should probably be refined
to take the SONAME from the Makefile somehow instead of hardcoding it.

Thanks and have a nice day!

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian Developer http://www.debian.org

Bruce Momjian wrote:
> Is this a direction we want to explore --- using the SONAME as part
> of the translation domain?

I think that's the way to go.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

I have developed the following patch which adds PG_VERSION to the end of
language-specific file names. I used it for libpq too, rather than the
SONAME, because the language strings might change between versions while
the API/SONAME might not.

Would someone who uses NLS please test this? Thanks.

---------------------------------------------------------------------------

Bruce Momjian wrote:
>
> Is this a direction we want to explore --- using the SONAME as part of
> the translation domain?
>
> ---------------------------------------------------------------------------
>
> Martin Pitt wrote:
> -- Start of PGP signed section.
> > Hi!
> >
> > Bruce Momjian [2005-02-09 18:05 -0500]:
> > > > However, I just stumbled across another problem: libpq3 and the new
> > > > libpq4 use the same translation domain "libpq4", thus they cannot be
> > > > installed in parallel. Can you please change the domain to "libpq4" as
> > > > well? This should generally be done anyway, but at least we can live
> > > > with one breakage of this rule (for libpq3), so that we don't need to
> > > > mess with the old libpq3 package.
> > >
> > > Uh, what is the translation domain?
> >
> > In short, that is the unique name for a bunch of translations for a
> > particular application or group of related applications. The key point
> > is that the name of the file in
> > /usr/share/locale/<LOCALE>/LC_MESSAGES/<DOMAIN>.mo and the call
> >
> > bindtextdomain ("<DOMAIN>", NULL)
> >
> > must use the same <DOMAIN> so that libintl can find the mo file.
> > However, that means that all applications that are installed in
> > parallel need a distinct domain. Since the whole point of SONAMes is
> > to allow several different library API versions to be installed in
> > parallel, every library API (i. e. SONAME) should have an unique
> > domain, which can be achieved easiest by just appending the SONAME to
> > the translation domain.
> >
> > In my test packages I used the following patch:
> >
> > diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c postgresql-8.0.1/src/interfaces/libpq/fe-misc.c
> > --- postgresql-8.0.1-old/src/interfaces/libpq/fe-misc.c 2004-12-31 23:03:50.000000000 +0100
> > +++ postgresql-8.0.1/src/interfaces/libpq/fe-misc.c 2005-02-07 22:55:13.177419296 +0100
> > @@ -1133,7 +1133,7 @@
> > {
> > already_bound = 1;
> > /* No relocatable lookup here because the binary could be anywhere */
> > - bindtextdomain("libpq", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
> > + bindtextdomain("libpq4", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);
> > }
> >
> > return dgettext("libpq", msgid);
> > diff -ruN postgresql-8.0.1-old/src/interfaces/libpq/nls.mk postgresql-8.0.1/src/interfaces/libpq/nls.mk
> > --- postgresql-8.0.1-old/src/interfaces/libpq/nls.mk 2005-01-14 09:57:06.000000000 +0100
> > +++ postgresql-8.0.1/src/interfaces/libpq/nls.mk 2005-02-07 22:54:54.770217616 +0100
> > @@ -1,5 +1,5 @@
> > # $PostgreSQL: pgsql/src/interfaces/libpq/nls.mk,v 1.20 2005/01/14 08:57:06 petere Exp $
> > -CATALOG_NAME := libpq
> > +CATALOG_NAME := libpq4
> > AVAIL_LANGUAGES := af cs de es fr hr it ko nb pl pt_BR ru sk sl sv tr zh_CN zh_TW
> > GETTEXT_FILES := fe-auth.c fe-connect.c fe-exec.c fe-lobj.c fe-misc.c fe-protocol2.c fe-protocol3.c fe-secure.c
> > GETTEXT_TRIGGERS:= libpq_gettext pqInternalNotice:2
> >
> > Compared to the SONAME, changing the translation domain is relatively
> > uncritical, so if you don't want to change this upstream, I can
> > maintain this patch for Debian/Ubuntu. However, I heard that some RPM
> > guys plan a infrastructure similar to mine, and at that point they
> > will have precisely the same problems :-)
> >
> > Thanks for considering and have a nice day!
> >
> > Martin
> > --
> > Martin Pitt http://www.piware.de
> > Ubuntu Developer http://www.ubuntulinux.org
> > Debian GNU/Linux Developer http://www.debian.org
> -- End of PGP section, PGP failed!
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania 19073
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> message can get through to the mailing list cleanly
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> I have developed the following patch which adds PG_VERSION to the end
> of language-specific file names.

In my mind, that would only make sense if we added the version number to
all program binaries as well (which we do not, of course). Otherwise,
what's the point?

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > I have developed the following patch which adds PG_VERSION to the end
> > of language-specific file names.
>
> In my mind, that would only make sense if we added the version number to
> all program binaries as well (which we do not, of course). Otherwise,
> what's the point?

The 'bind' calles in the binaries are going to look for the proper
version. Does that help, or is libpq the only thing we need to handle?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> The 'bind' calles in the binaries are going to look for the proper
> version. Does that help, or is libpq the only thing we need to
> handle?

Shared libraries have their version number embedded in the file name for
the explicit purpose of installing more than one version side by side.
Therefore, the PO files also need to make arrangements for side by side
installation. No such promises are made for non-library files.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > The 'bind' calles in the binaries are going to look for the proper
> > version. Does that help, or is libpq the only thing we need to
> > handle?
>
> Shared libraries have their version number embedded in the file name for
> the explicit purpose of installing more than one version side by side.
> Therefore, the PO files also need to make arrangements for side by side
> installation. No such promises are made for non-library files.

OK, seems we need examples of how our current setup fails, both for
libpq and binaries.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Hi!

Bruce Momjian [2005-06-15 15:26 -0400]:
> Peter Eisentraut wrote:
> > Bruce Momjian wrote:
> > > The 'bind' calles in the binaries are going to look for the proper
> > > version. Does that help, or is libpq the only thing we need to
> > > handle?
> >
> > Shared libraries have their version number embedded in the file name for
> > the explicit purpose of installing more than one version side by side.
> > Therefore, the PO files also need to make arrangements for side by side
> > installation. No such promises are made for non-library files.
>
> OK, seems we need examples of how our current setup fails, both for
> libpq and binaries.

Debian's and Ubuntu's packages currently look like this (binary ->
translation domain):

/usr/lib/libpq3.so: libpq3
/usr/lib/libpq4.so: libpq4
/usr/lib/postgresql/7.4/bin/postmaster: postgres-7.4
/usr/lib/postgresql/8.0/bin/postmaster: postgres-8.0
/usr/lib/postgresql/7.4/bin/psql: psql-7.4
/usr/lib/postgresql/8.0/bin/psql: psql-8.0
[similar for all other client binaries)

Without my domain patches, i. e. with upstream's scheme it would look
like:

/usr/lib/libpq3.so: libpq
/usr/lib/libpq4.so: libpq
/usr/lib/postgresql/7.4/bin/postmaster: postgres
/usr/lib/postgresql/8.0/bin/postmaster: postgres

As you see, there is a conflict, so to be able to install both
pacakges at the same time, one translation version had to be picked
and stuffed into a separate -translations package, which both versions
depend on. That's ugly and would cause one version to have some
missing translations, therefore I patched the domains to be
version-specific.

I do think that adopting my scheme at least for libpq really makes
sense. Adopting it for the binaries would not do any harm, but it
might not be conformant to your packaging policy, which I don't really
know. The clashing domains are the only things that prevents the
parallel installation of different major versions, so the question
whether or not you want to adopt it mainly boils down to whether you
want to support parallel server installations.

I am grateful that you did the SONAME change upstream, that relieved
Debian and probably other vendors of much pain. OTOH, the Debian
specific translation domain changes are really simple and don't impose
a significant maintenance overhead, so there is no urgency at all.

Thanks and have a nice day!

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntu.com
Debian Developer http://www.debian.org

Martin Pitt wrote:
> /usr/lib/libpq3.so: libpq3
> /usr/lib/libpq4.so: libpq4
> /usr/lib/postgresql/7.4/bin/postmaster: postgres-7.4
> /usr/lib/postgresql/8.0/bin/postmaster: postgres-8.0
> /usr/lib/postgresql/7.4/bin/psql: psql-7.4
> /usr/lib/postgresql/8.0/bin/psql: psql-8.0
> [similar for all other client binaries)
>
> Without my domain patches, i. e. with upstream's scheme it would look
> like:
>
> /usr/lib/libpq3.so: libpq
> /usr/lib/libpq4.so: libpq
> /usr/lib/postgresql/7.4/bin/postmaster: postgres
> /usr/lib/postgresql/8.0/bin/postmaster: postgres
>
> As you see, there is a conflict, so to be able to install both
> pacakges at the same time, one translation version had to be picked
> and stuffed into a separate -translations package, which both versions
> depend on. That's ugly and would cause one version to have some
> missing translations, therefore I patched the domains to be
> version-specific.
>
> I do think that adopting my scheme at least for libpq really makes
> sense. Adopting it for the binaries would not do any harm, but it
> might not be conformant to your packaging policy, which I don't really
> know. The clashing domains are the only things that prevents the
> parallel installation of different major versions, so the question
> whether or not you want to adopt it mainly boils down to whether you
> want to support parallel server installations.
>
> I am grateful that you did the SONAME change upstream, that relieved
> Debian and probably other vendors of much pain. OTOH, the Debian
> specific translation domain changes are really simple and don't impose
> a significant maintenance overhead, so there is no urgency at all.

We absolutely want to support multiple installed versions of PostgreSQL.
In my understanding, it isn't just libpq that depends on
version-specific translation files, but all binaries that have
translations.

Also, it is possible for the SONAME not to change between major versions
(if the libpq API remains unchanged), but for the translations to
change.

For this reason, I think we should add the major number to all
bindtextdomain() calls, as is done in the attached patch.

Is this conclusion correct?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Am Samstag, 25. Juni 2005 04:24 schrieb Bruce Momjian:
> We absolutely want to support multiple installed versions of PostgreSQL.

But we don't support installing multiple versions on top of each other, which
is the only scenario where this patch would be useful.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Am Samstag, 25. Juni 2005 04:24 schrieb Bruce Momjian:
>> We absolutely want to support multiple installed versions of PostgreSQL.

> But we don't support installing multiple versions on top of each other, which
> is the only scenario where this patch would be useful.

I agree with Peter. AFAICS, the binaries go into $prefix/bin and the
NLS files go into $prefix/share/locale, so there is no more nor less
conflict among different versions' NLS files than their executables.
In both cases the only supported way to have multiple installations
at once is to use different install prefixes. So why complicate matters
like this?

regards, tom lane

Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > Am Samstag, 25. Juni 2005 04:24 schrieb Bruce Momjian:
> >> We absolutely want to support multiple installed versions of PostgreSQL.
>
> > But we don't support installing multiple versions on top of each other, which
> > is the only scenario where this patch would be useful.
>
> I agree with Peter. AFAICS, the binaries go into $prefix/bin and the
> NLS files go into $prefix/share/locale, so there is no more nor less
> conflict among different versions' NLS files than their executables.
> In both cases the only supported way to have multiple installations
> at once is to use different install prefixes. So why complicate matters
> like this?

OK. I guess I am confused.

Right now bindtextdomain() just calls something like
bindtextdomain("libpq"), etc. How does it know to use the proper
share/locale? And is bindtextdomain("psql") different?

I am just trying to understand how things work.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Hi!

Peter Eisentraut [2005-06-25 11:29 +0200]:
> Am Samstag, 25. Juni 2005 04:24 schrieb Bruce Momjian:
> > We absolutely want to support multiple installed versions of PostgreSQL.
>
> But we don't support installing multiple versions on top of each other, which
> is the only scenario where this patch would be useful.

It's not "on top of each other", but side-by-side as Bruce says. The
single clusters and their postmaster do not influence each other in
Debian.

Martin

--
Martin Pitt http://www.piware.de
Ubuntu Developer http://www.ubuntulinux.org
Debian Developer http://www.debian.org

Martin Pitt wrote:
> Hi!
>
> Peter Eisentraut [2005-06-25 11:29 +0200]:
> > Am Samstag, 25. Juni 2005 04:24 schrieb Bruce Momjian:
> > > We absolutely want to support multiple installed versions of PostgreSQL.
> >
> > But we don't support installing multiple versions on top of each other, which
> > is the only scenario where this patch would be useful.
>
> It's not "on top of each other", but side-by-side as Bruce says. The
> single clusters and their postmaster do not influence each other in
> Debian.

I have looked now at the bindtextdomain() code in current CVS and 8.0.X.

We basically have only two bindtextdomain() calls. libpq has this code:

bindtextdomain("libpq", getenv("PGLOCALEDIR") ? getenv("PGLOCALEDIR") : LOCALEDIR);

It basically uses either the environment variable PGLOCALEDIR, or the
hard-coded compile-time LOCALEDIR value. I don't see how adding a
version number to the "libpq" string would help unles PGLOCALEDIR points
to a directory that has both libpq-7.4 and libpq-8.0 directories with
translations, and I didn't see your setup using that.

Our applications call set_pglocale_pgservice(), which makes two
bindtextdomain() calls. It calls set_pglocale_pgservice(), like this:

set_pglocale_pgservice(argv[0], "psql");

The locale files for the application are found relative to the directory
containing the binary. It also sets the environment variable
PGLOCALEDIR so libpq can find its local files.

Based on this analsys, I don't understand what adding the PostgreSQL
version number does for us.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073