PQnotifies() in 7.3 broken?

Thanks to Rod Taylor's kind help in donating a system account, I've been
able to test libpqxx against postgres 7.3. Unfortunately, I'm running
into several problems. One thing that broke libpqxx was a change in
cursor behaviour that according to Sigurdur Gunnlaugsson seems to be
gone in the 7.4 development tree. But I'm hitting other snags as well.

When receiving a trigger notification under 7.3, the structure returned
by PQnotifies() appears to be bogus. In a test I ran, its be_pid was
consistently zero and its relname pointed into never-neverland.

Has anyone else come across this?

Jeroen

"Jeroen T. Vermeulen" <jtv(at)xs4all(dot)nl> writes:
> When receiving a trigger notification under 7.3, the structure returned
> by PQnotifies() appears to be bogus. In a test I ran, its be_pid was
> consistently zero and its relname pointed into never-neverland.

We changed the PQnotifies result structure in 7.3 (to insulate clients
from the value of NAMEDATALEN). I think you are compiling your app with
a 7.3 libpq header and then running it with 7.2 libpq code, or possibly
vice versa.

regards, tom lane

Perhaps the .so name should have been updated for PostgreSQL 7.3? For
example in 7.2 libpq is:

/usr/lib/libpq.so -> libpq.so.2.2
/usr/lib/libpq.so.2 -> libpq.so.2.2
/usr/lib/libpq.so.2.0 -> libpq.so.2
/usr/lib/libpq.so.2.2

and PostgreSQL 7.3:

/usr/lib/libpq.so -> libpq.so.2.2
/usr/lib/libpq.so.2 -> libpq.so.2.2
/usr/lib/libpq.so.2.0 -> libpq.so.2
/usr/lib/libpq.so.2.2

the same. This would seem to imply binary compatibility?

Lee.

Tom Lane writes:
> "Jeroen T. Vermeulen" <jtv(at)xs4all(dot)nl> writes:
> > When receiving a trigger notification under 7.3, the structure returned
> > by PQnotifies() appears to be bogus. In a test I ran, its be_pid was
> > consistently zero and its relname pointed into never-neverland.
> We changed the PQnotifies result structure in 7.3 (to insulate clients
> from the value of NAMEDATALEN). I think you are compiling your app with
> a 7.3 libpq header and then running it with 7.2 libpq code, or possibly
> vice versa.

Lee Kindness <lkindness(at)csl(dot)co(dot)uk> writes:
> Perhaps the .so name should have been updated for PostgreSQL 7.3?

It should have been. If it wasn't, that was a serious oversight.
Not sure if we should change it in 7.3.1 or not, though; it may be
too late for that. Any thoughts out there?

regards, tom lane

Tom Lane wrote:
> Lee Kindness <lkindness(at)csl(dot)co(dot)uk> writes:
> > Perhaps the .so name should have been updated for PostgreSQL 7.3?
>
> It should have been. If it wasn't, that was a serious oversight.
> Not sure if we should change it in 7.3.1 or not, though; it may be
> too late for that. Any thoughts out there?

Seems I did forget. I always update the minor for a major release, but
when development starts, and I seem to have forgotten for 7.3. Sorry.

I will update for 7.4 now. Too late for 7.3 clearly.

Turns out I usually do it when I stamp the new development tree, but
someone else stamped 7.3 and 7.4. :-(

Here is 7.2 stamp, which shows the updates:

revision 1.52
date: 2001/05/11 01:46:33; author: momjian; state: Exp; lines: +2 -2
Stamp CVS as 7.2. Update all interface version numbers. This is the
time to do it, not during beta because people are using this stuff in
production sometimes.

The diff shows:

***************
*** 15,21 ****
# shared library parameters
NAME= pq
SO_MAJOR_VERSION= 2
! SO_MINOR_VERSION= 1

override CPPFLAGS := -I$(srcdir) $(CPPFLAGS) -DFRONTEND
-DSYSCONFDIR='"$(sysco
nfdir)"'

--- 15,21 ----
# shared library parameters
NAME= pq
SO_MAJOR_VERSION= 2
! SO_MINOR_VERSION= 2

so clearly 7.2 and 7.3 have the same minor version for all interfaces. Bad!

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian writes:

> so clearly 7.2 and 7.3 have the same minor version for all interfaces. Bad!

We forgot between 7.0 and 7.1 as well, so it's at least consistent...

--
Peter Eisentraut peter_e(at)gmx(dot)net

Peter Eisentraut wrote:
> Bruce Momjian writes:
>
> > so clearly 7.2 and 7.3 have the same minor version for all interfaces. Bad!
>
> We forgot between 7.0 and 7.1 as well, so it's at least consistent...

Yes, seems we increament on every even-numbered release. ;-)

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Wed, 2002-12-04 at 13:11, Bruce Momjian wrote:
> Seems I did forget. I always update the minor for a major release, but
> when development starts, and I seem to have forgotten for 7.3. Sorry.
>
> I will update for 7.4 now. Too late for 7.3 clearly.

Wouldn't that suggest that libpq in 7.4 and 7.3 are *not* binary
compatible? AFAIK that's not the case...

Cheers,

Neil
--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC

Neil Conway wrote:
> On Wed, 2002-12-04 at 13:11, Bruce Momjian wrote:
> > Seems I did forget. I always update the minor for a major release, but
> > when development starts, and I seem to have forgotten for 7.3. Sorry.
> >
> > I will update for 7.4 now. Too late for 7.3 clearly.
>
> Wouldn't that suggest that libpq in 7.4 and 7.3 are *not* binary
> compatible? AFAIK that's not the case...

I updated the _minor_ version number, which doesn't affect binary
compatibility, I think. It only makes the newer one more visible.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian writes:
> Tom Lane wrote:
> > Lee Kindness <lkindness(at)csl(dot)co(dot)uk> writes:
> > > Perhaps the .so name should have been updated for PostgreSQL 7.3?
> >
> > It should have been. If it wasn't, that was a serious oversight.
> > Not sure if we should change it in 7.3.1 or not, though; it may be
> > too late for that. Any thoughts out there?
> Seems I did forget. I always update the minor for a major release, but
> when development starts, and I seem to have forgotten for 7.3. Sorry.

Personally I think automatically updating the version numbers is as
bad as not doing it at all - it misses the point.

The version numbers in shared library filenames denote binary
compatibilty, if the /public/ API changes then the version number
really must be incremented.

If the version increments without any associated API changes then it's
just a PITA for developers and products linking to the PostgreSQL
libraries! It forces recompilation when there is not really a need.

Lee.

Lee Kindness wrote:
> Bruce Momjian writes:
> > Tom Lane wrote:
> > > Lee Kindness <lkindness(at)csl(dot)co(dot)uk> writes:
> > > > Perhaps the .so name should have been updated for PostgreSQL 7.3?
> > >
> > > It should have been. If it wasn't, that was a serious oversight.
> > > Not sure if we should change it in 7.3.1 or not, though; it may be
> > > too late for that. Any thoughts out there?
> > Seems I did forget. I always update the minor for a major release, but
> > when development starts, and I seem to have forgotten for 7.3. Sorry.
>
> Personally I think automatically updating the version numbers is as
> bad as not doing it at all - it misses the point.
>
> The version numbers in shared library filenames denote binary
> compatibilty, if the /public/ API changes then the version number
> really must be incremented.
>
> If the version increments without any associated API changes then it's
> just a PITA for developers and products linking to the PostgreSQL
> libraries! It forces recompilation when there is not really a need.

It is my understanding that only the major number force recompliation.
We came up with this procedure after quite a bit of discussion. I am
happy to change it, but I need more than one person to tell me that.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
>
> I will update for 7.4 now. Too late for 7.3 clearly.
>

Bruce, why is it too late?

Most (all) will upgrade to 7.3.1 anyway, so it is a chance to get things right.

--
Fernando Nasser
Red Hat - Toronto E-Mail: fnasser(at)redhat(dot)com
2323 Yonge Street, Suite #300
Toronto, Ontario M4P 2C9

Fernando Nasser wrote:
> Bruce Momjian wrote:
> >
> > I will update for 7.4 now. Too late for 7.3 clearly.
> >
>
> Bruce, why is it too late?
>
> Most (all) will upgrade to 7.3.1 anyway, so it is a chance to get things right.

Oh. yes. Is it safe to do that?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Fernando Nasser wrote:
>> Bruce Momjian wrote:
>> Bruce, why is it too late?
>>
>> Most (all) will upgrade to 7.3.1 anyway, so it is a chance to get things right.

> Oh. yes. Is it safe to do that?

The RPM packagers should probably have a say in this, but I'm leaning
to agree with Fernando. The bulk of our users probably will not update
from 7.2.* to 7.3 before 7.3.1 is out (at least not for production),
so the fact that 7.3 has the wrong library version number won't affect
them. But people will continue to get burnt if we don't fix it for
7.3.1.

It is not real clear to me whether we need a major version bump, rather
than a minor one. We *do* need to signal binary incompatibility. Who
can clarify the rules here?

regards, tom lane

On Thu, Dec 05, 2002 at 03:27:23PM -0500, Tom Lane wrote:
>
> It is not real clear to me whether we need a major version bump, rather
> than a minor one. We *do* need to signal binary incompatibility. Who
> can clarify the rules here?

One thing I wonder about: should the rules make any distinction between
API incompatibilities and client protocol incompatibilities? For the
former I would imagine one would like to have some "minor" version number
increase whenever features are added and a "major" number be incremented
when changes become incompatible. For the former, one would probably
want to have a similar rule but with a dichotomy between server-side
upgrades and client-side upgrades.

Jeroen

Jeroen T. Vermeulen wrote:
> On Thu, Dec 05, 2002 at 03:27:23PM -0500, Tom Lane wrote:
> >
> > It is not real clear to me whether we need a major version bump, rather
> > than a minor one. We *do* need to signal binary incompatibility. Who
> > can clarify the rules here?
>
> One thing I wonder about: should the rules make any distinction between
> API incompatibilities and client protocol incompatibilities? For the
> former I would imagine one would like to have some "minor" version number
> increase whenever features are added and a "major" number be incremented
> when changes become incompatible. For the former, one would probably
> want to have a similar rule but with a dichotomy between server-side
> upgrades and client-side upgrades.

Yes, now that I remember, that was the big distinction. One requires a
recompile, the other one doesn't even work with a newer db.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Tom Lane writes:

> It is not real clear to me whether we need a major version bump, rather
> than a minor one. We *do* need to signal binary incompatibility. Who
> can clarify the rules here?

Strictly speaking, it's platform-dependent, but our shared library code
plays a bit of abuse with it. What it comes down to is:

If you change or remove an interface, increment the major version number.
If you add an interface, increment the minor version number. If you did
neither but changed the source code at all, increment the third version
number, if we had one.

To be thoroughly amused, read the libtool source. Grep for 'version_type'.

--
Peter Eisentraut peter_e(at)gmx(dot)net

Peter Eisentraut wrote:
> Tom Lane writes:
>
> > It is not real clear to me whether we need a major version bump, rather
> > than a minor one. We *do* need to signal binary incompatibility. Who
> > can clarify the rules here?
>
> Strictly speaking, it's platform-dependent, but our shared library code
> plays a bit of abuse with it. What it comes down to is:
>
> If you change or remove an interface, increment the major version number.
> If you add an interface, increment the minor version number. If you did
> neither but changed the source code at all, increment the third version
> number, if we had one.

My take on it is: if you break binary compatibility, which includes
semantic dependencies, then you increment the major version number.
Otherwise you increment the minor version number.

Since the change we're talking about apparently broke binary
compatibility, it calls for a major version number change.

Protocol incompatibility requires a different method of enforcement
altogether. Incrementing the major number of the library isn't going
to do you any good for that (7.2 clients on remote systems won't have
any idea that the libraries on the 7.3 server have changed that much).

All MHO, of course...

--
Kevin Brown kevin(at)sysexperts(dot)com
This is your .signature virus: < begin 644 .signature (9V]T8VAA(0K0z end >
This is your .signature virus on drugs: <>
Any questions?

OK, seeing that we don't have a third number, do people want me to
increment the interface numbers for 7.3.1, or just wait for the
increment in 7.4?

---------------------------------------------------------------------------

Peter Eisentraut wrote:
> Tom Lane writes:
>
> > It is not real clear to me whether we need a major version bump, rather
> > than a minor one. We *do* need to signal binary incompatibility. Who
> > can clarify the rules here?
>
> Strictly speaking, it's platform-dependent, but our shared library code
> plays a bit of abuse with it. What it comes down to is:
>
> If you change or remove an interface, increment the major version number.
> If you add an interface, increment the minor version number. If you did
> neither but changed the source code at all, increment the third version
> number, if we had one.
>
> To be thoroughly amused, read the libtool source. Grep for 'version_type'.
>
> --
> Peter Eisentraut peter_e(at)gmx(dot)net
>
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

At 06:36 PM 10/12/2002 -0500, Bruce Momjian wrote:
>do people want me to
>increment the interface numbers for 7.3.1

I'd like it because I have to support & build against multiple versions.

----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.B.N. 75 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 03 5330 3172 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/

Seems like a mistake was made. Let's (don't ya love how that sounds
like I'm actually involved in the fix? ;) fix it sooner rather than
later.

Just curious, after a release, how come the numbers are not
automatically bumped to ensure this type thing gets caught sooner rather
than later? Is it possible to automate this as part of the build
process so that they get grabbed from some version information during
the build?

Greg

On Tue, 2002-12-10 at 17:36, Bruce Momjian wrote:
> OK, seeing that we don't have a third number, do people want me to
> increment the interface numbers for 7.3.1, or just wait for the
> increment in 7.4?
>
> ---------------------------------------------------------------------------
>
> Peter Eisentraut wrote:
> > Tom Lane writes:
> >
> > > It is not real clear to me whether we need a major version bump, rather
> > > than a minor one. We *do* need to signal binary incompatibility. Who
> > > can clarify the rules here?
> >
> > Strictly speaking, it's platform-dependent, but our shared library code
> > plays a bit of abuse with it. What it comes down to is:
> >
> > If you change or remove an interface, increment the major version number.
> > If you add an interface, increment the minor version number. If you did
> > neither but changed the source code at all, increment the third version
> > number, if we had one.
> >
> > To be thoroughly amused, read the libtool source. Grep for 'version_type'.
> >
> > --
> > Peter Eisentraut peter_e(at)gmx(dot)net
> >
> >
--
Greg Copeland <greg(at)copelandconsulting(dot)net>
Copeland Computer Consulting

Greg Copeland wrote:
> Seems like a mistake was made. Let's (don't ya love how that sounds
> like I'm actually involved in the fix? ;) fix it sooner rather than
> later.
>
> Just curious, after a release, how come the numbers are not
> automatically bumped to ensure this type thing gets caught sooner rather
> than later? Is it possible to automate this as part of the build
> process so that they get grabbed from some version information during
> the build?

Version bump is one of the few things we do at the start of development.
For 7.2, I didn't actually stamp the 7.2 release so I never bumped them,
or I forgot. Seems I also forgot for 7.1. It is listed in
tools/RELEASE_CHANGES so it is just a matter of following that file.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Greg Copeland wrote:
>> Is it possible to automate this as part of the build
>> process so that they get grabbed from some version information during
>> the build?

> Version bump is one of the few things we do at the start of
> development.

The real problem here is that major version bump (signifying an
incompatible API change) is something that must NOT be done in an
automated, mindless-checklist way. We should have executed the bump
when we agreed to change PQnotifies' API incompatibly. We screwed up
on that. I think it's correct to fix the error for 7.3.1 --- but we
cannot improve on the situation by making some procedural change to
"always do X at point Y in the release cycle". Sometimes there's
no substitute for actual thinking :-(

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Greg Copeland wrote:
> >> Is it possible to automate this as part of the build
> >> process so that they get grabbed from some version information during
> >> the build?
>
> > Version bump is one of the few things we do at the start of
> > development.
>
> The real problem here is that major version bump (signifying an
> incompatible API change) is something that must NOT be done in an
> automated, mindless-checklist way. We should have executed the bump
> when we agreed to change PQnotifies' API incompatibly. We screwed up
> on that. I think it's correct to fix the error for 7.3.1 --- but we
> cannot improve on the situation by making some procedural change to
> "always do X at point Y in the release cycle". Sometimes there's
> no substitute for actual thinking :-(

Oh, a major bump. I thought we did major bumps only in cases where a
recompile will _not_ fix the problem, like changing a parameter value to
a function or removing a function or something like that.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

I have bumped minor versions for 7.3 and 7.4. If we decide to do
something different later, fine, but this way we will not forget to have
some update for 7.3.

---------------------------------------------------------------------------

Bruce Momjian wrote:
> Tom Lane wrote:
> > Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > > Greg Copeland wrote:
> > >> Is it possible to automate this as part of the build
> > >> process so that they get grabbed from some version information during
> > >> the build?
> >
> > > Version bump is one of the few things we do at the start of
> > > development.
> >
> > The real problem here is that major version bump (signifying an
> > incompatible API change) is something that must NOT be done in an
> > automated, mindless-checklist way. We should have executed the bump
> > when we agreed to change PQnotifies' API incompatibly. We screwed up
> > on that. I think it's correct to fix the error for 7.3.1 --- but we
> > cannot improve on the situation by making some procedural change to
> > "always do X at point Y in the release cycle". Sometimes there's
> > no substitute for actual thinking :-(
>
> Oh, a major bump. I thought we did major bumps only in cases where a
> recompile will _not_ fix the problem, like changing a parameter value to
> a function or removing a function or something like that.
>
> --
> Bruce Momjian | http://candle.pha.pa.us
> pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
> + If your life is a hard drive, | 13 Roberts Road
> + Christ can be your backup. | Newtown Square, Pennsylvania 19073
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/users-lounge/docs/faq.html
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> Tom Lane wrote:
> > Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > > Greg Copeland wrote:
> > >> Is it possible to automate this as part of the build
> > >> process so that they get grabbed from some version information during
> > >> the build?
> >
> > > Version bump is one of the few things we do at the start of
> > > development.
> >
> > The real problem here is that major version bump (signifying an
> > incompatible API change) is something that must NOT be done in an
> > automated, mindless-checklist way. We should have executed the bump
> > when we agreed to change PQnotifies' API incompatibly. We screwed up
> > on that. I think it's correct to fix the error for 7.3.1 --- but we
> > cannot improve on the situation by making some procedural change to
> > "always do X at point Y in the release cycle". Sometimes there's
> > no substitute for actual thinking :-(
>
> Oh, a major bump. I thought we did major bumps only in cases where a
> recompile will _not_ fix the problem, like changing a parameter value to
> a function or removing a function or something like that.

That's not strictly how the major and minor numbers were intended to
be used, at least as I understand it.

The reason you really have no choice but to bump the major number in
the case of the introduction of binary incompatibilities (whether or
not a recompile would fix it) is that the dynamic linker usually uses
the major version *only* to determine which library to dynamically
link against (but see below for a caveat).

So what's the purpose of the minor version number? To indicate which
revision of the library is in use. It may be that version 2.1 has
bugfixes that 2.0 doesn't have, so the minor version number allows you
to determine whether or not you have the fixes in question.

Generally, you can specify at library build time whether an
application must link with a specific major/minor numbered library, or
whether it can link against any library with the same major number.
Most people do the latter, and that's as it should be. If the
PostgreSQL libraries (for instance) required a match against the minor
number, then applications would have to be recompiled every time
PostgreSQL was upgraded. Not only is that highly undesirable, for
some it may not even be possible (e.g., when using commercial
applications).

--
Kevin Brown kevin(at)sysexperts(dot)com

Bruce Momjian writes:

> OK, seeing that we don't have a third number, do people want me to
> increment the interface numbers for 7.3.1, or just wait for the
> increment in 7.4?

ISTM that the briefly established strategy to bump the version numbers at
the beginning of development is not satisfactory. The reason is that the
"beginning" is in many cases not well-defined. Example 1: If we hadn't
noticed the PQnotifies() problem that started this thread we would have
forgotten again. Example 2: If someone put a fix of some sort in libpq
for 7.3.1, we would surely forget to bump the version number.

Consequence: The library version number must be bumped as part of the
release preparation, as is in fact written down in the release checklist.
And such a bump should be done only after reviewing the change history of
the library during that development cycle to determine if a major bump is
necessary or if in fact no change at all is warranted.

As for what to do right now, if others agree I would suggest that we do
the appropriate version number adjustments (as described in the previous
paragraph) for 7.3.1.

--
Peter Eisentraut peter_e(at)gmx(dot)net

Peter Eisentraut wrote:
> Bruce Momjian writes:
>
> > OK, seeing that we don't have a third number, do people want me to
> > increment the interface numbers for 7.3.1, or just wait for the
> > increment in 7.4?
>
> ISTM that the briefly established strategy to bump the version numbers at
> the beginning of development is not satisfactory. The reason is that the

We bump at the beginning only because we _know_ we want new users to use
the newer library. (Does the runtime linker know to get the highest
minor numbered library with the same major number?)

Bumping at the end is done only when we know there is some change. The
big question is whether a change in the API or a change in the code
(recompile) is enough to bump that major version number. We always make
some force-recompile change in the library in each release, don't we?
Do we just bump the major in every major release?

> "beginning" is in many cases not well-defined. Example 1: If we hadn't
> noticed the PQnotifies() problem that started this thread we would have
> forgotten again. Example 2: If someone put a fix of some sort in libpq
> for 7.3.1, we would surely forget to bump the version number.
>
> Consequence: The library version number must be bumped as part of the
> release preparation, as is in fact written down in the release checklist.

I usually bumped the minor at the beginning because this allows beta
testers to not have _extra_ versions of the library laying around, and
also because we make minor library changes often during beta, so it
isn't clear when to bump that number.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> We bump at the beginning only because we _know_ we want new users to use
> the newer library. (Does the runtime linker know to get the highest
> minor numbered library with the same major number?)

It probably depends on the system, but the runtime linker isn't that
smart under Linux. It looks for a match for the major version only,
so for instance in the case of libpq major version 2, it'll look for
libpq.so.2 in the library search path. Multiple minor versions of the
library are managed via symlinks under Linux (libpq.so.2 ->
libpq.so.2.2, for instance).

> Bumping at the end is done only when we know there is some change. The
> big question is whether a change in the API or a change in the code
> (recompile) is enough to bump that major version number. We always make
> some force-recompile change in the library in each release, don't we?
> Do we just bump the major in every major release?

It wouldn't be a terribly bad idea to do that, but the main criteria
for bumping the major version should be binary compatibility. If
applications which link against libpq.so.2 reside on the system and
libpq.so.2.3 has binary incompatibilities with libpq.so.2.2, then
installing libpq.so.2.3 on the system will break all the binaries that
use libpq.so.2. That's why it's important to bump the major version
when there are binary incompatibilities: you can install libpq.so.3
and all the while, applications that rely on libpq.so.2 will still run
(because you can have both of those library versions installed on the
system at the same time without conflict).

> I usually bumped the minor at the beginning because this allows beta
> testers to not have _extra_ versions of the library laying around, and
> also because we make minor library changes often during beta, so it
> isn't clear when to bump that number.

I think it makes sense to change the minor number whenever there are
code changes to the library that don't introduce binary
incompatibilities. Whether you bump the minor version during a new
release when there are no changes to the library itself is probably a
matter of preference only. It doesn't really hurt anything and may
make management of the version number easier.

--
Kevin Brown kevin(at)sysexperts(dot)com

Kevin Brown wrote:
> It wouldn't be a terribly bad idea to do that, but the main criteria
> for bumping the major version should be binary compatibility. If
> applications which link against libpq.so.2 reside on the system and
> libpq.so.2.3 has binary incompatibilities with libpq.so.2.2, then
> installing libpq.so.2.3 on the system will break all the binaries that
> use libpq.so.2. That's why it's important to bump the major version
> when there are binary incompatibilities: you can install libpq.so.3
> and all the while, applications that rely on libpq.so.2 will still run
> (because you can have both of those library versions installed on the
> system at the same time without conflict).
>
> > I usually bumped the minor at the beginning because this allows beta
> > testers to not have _extra_ versions of the library laying around, and
> > also because we make minor library changes often during beta, so it
> > isn't clear when to bump that number.
>
> I think it makes sense to change the minor number whenever there are
> code changes to the library that don't introduce binary
> incompatibilities. Whether you bump the minor version during a new
> release when there are no changes to the library itself is probably a
> matter of preference only. It doesn't really hurt anything and may
> make management of the version number easier.

If it is true that the linker only matches the major number, what value
is there in incrementing the minor number, as we have done in the past?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
> If it is true that the linker only matches the major number, what value
> is there in incrementing the minor number, as we have done in the
> past?

It's main value is in indicating to the system administrator which
version of the library he has. This is particularly useful in the
face of security updates.

A library can have bugfixes (even significant ones) and still be
binary compatible with a previous revision. Situations like that are
what the minor number is useful for. It can make the developer's (or
administrator's) life a little easier, since it means he can have
multiple minor revisions installed on his system and change the major
revision symlink to point to whichever one he wishes to actually use
(given the way the dynamic linker works, the administrator can name
the actual target of the symlink whatever he wishes so it's not
mandatory that the developers do that for him...it just makes his life
easier).

I probably should have gone into a little more detail about how the
linker does its thing: it looks for a literal match for what the
application says it was compiled against. When the application was
linked, if the library it was linked against said it was 'libpg.so.2'
(As far as I know, the compile-time linker retrieves this from the
library's header information, not from its filename), then that's what
gets stored in the application's executable header and is what the
runtime linker looks for. libpg.so.2 may be a symlink to (e.g.)
libpg.so.2.1 or a hard link, or a copy of a library for that matter.
The runtime linker will successfully link the shared library into the
program's memory image in any of those cases -- it only has to find a
shared library whose filename is the same as what's requested *and*
whose header information provides the same name.

--
Kevin Brown kevin(at)sysexperts(dot)com

Bruce Momjian writes:

> We bump at the beginning only because we _know_ we want new users to use
> the newer library. (Does the runtime linker know to get the highest
> minor numbered library with the same major number?)

No, the run-time linker only looks at the major version.

> Bumping at the end is done only when we know there is some change.

That is the theory. But the practice is that we should, as you suggest,
reconsider the issue at the end but we don't because we all think we
already did it at the beginning. That considerably confuses the issue for
no real gain. During the development cycle the version number is pretty
irrelevant because it is more likely that the code is broken than that
there is a genuine library conflict that could be usefully resolved.

> The big question is whether a change in the API or a change in the code
> (recompile) is enough to bump that major version number. We always make
> some force-recompile change in the library in each release, don't we?
> Do we just bump the major in every major release?

The rules are pretty clear: If you change or remove an interface (= part
of the API) then you change the major, if you add an interface or
"improve" the code, change the minor. Whether we actually change the code
during a release cycle is something that is best determined *after* the
release cycle. Possibly we always do, but I wouldn't be surprised if some
library like pgeasy lay dormant for a while.

> I usually bumped the minor at the beginning because this allows beta
> testers to not have _extra_ versions of the library laying around, and

That doesn't make sense to me. Which extra versions?

--
Peter Eisentraut peter_e(at)gmx(dot)net

Peter Eisentraut wrote:
> Bruce Momjian writes:
>
> > We bump at the beginning only because we _know_ we want new users to use
> > the newer library. (Does the runtime linker know to get the highest
> > minor numbered library with the same major number?)
>
> No, the run-time linker only looks at the major version.

Then what value is there to incrementing the minor number?

> > The big question is whether a change in the API or a change in the code
> > (recompile) is enough to bump that major version number. We always make
> > some force-recompile change in the library in each release, don't we?
> > Do we just bump the major in every major release?
>
> The rules are pretty clear: If you change or remove an interface (= part
> of the API) then you change the major, if you add an interface or
> "improve" the code, change the minor. Whether we actually change the code

So if a recompile fixes it, increment minor, else major. Then we
normally only do minor-level changes,. and frankly we improve the code
all during development time and during beta, so it seems pretty abitrary
when we increment the minor unless we want to increment it many times
during development, _or_ right before final to ensure that final
releaase users have the newest version.

> during a release cycle is something that is best determined *after* the
> release cycle. Possibly we always do, but I wouldn't be surprised if some
> library like pgeasy lay dormant for a while.
>
> > I usually bumped the minor at the beginning because this allows beta
> > testers to not have _extra_ versions of the library laying around, and
>
> That doesn't make sense to me. Which extra versions?

If you bump during beta, and don't delete your pgsql/ directory, you
will have the old libpq.so there and the new one because once you bump
the version, the old one just says in the directory.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> So if a recompile fixes it, increment minor, else major.

Wrong --- if you need a recompile then it's not binary-compatible, so
it should be a major version bump.

> Then we
> normally only do minor-level changes,. and frankly we improve the code
> all during development time and during beta, so it seems pretty abitrary
> when we increment the minor unless we want to increment it many times
> during development, _or_ right before final to ensure that final
> releaase users have the newest version.

I think there is definite value in incrementing the minor version once
at the start of the cycle. Whether we do it at start or end does not
matter to end users, but it *does* help developers, who can then easily
tell whether they are looking at a devel library or the previous
release.

If we make a non-binary-compatible change during a development cycle,
we should then immediately bump the major version number. Leaving it
till the end of the cycle is an excellent recipe for forgetting, as
indeed we just did.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > So if a recompile fixes it, increment minor, else major.
>
> Wrong --- if you need a recompile then it's not binary-compatible, so
> it should be a major version bump.

But the previous poster said only API changes were reasons to bump the
major, right?
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Tom Lane wrote:
>> Wrong --- if you need a recompile then it's not binary-compatible, so
>> it should be a major version bump.

> But the previous poster said only API changes were reasons to bump the
> major, right?

Yes. He meant *binary* API changes, though, ie, anything that would
break extant executables originally linked to the prior version of the
shared library.

regards, tom lane

Tom Lane wrote:
> Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > Tom Lane wrote:
> >> Wrong --- if you need a recompile then it's not binary-compatible, so
> >> it should be a major version bump.
>
> > But the previous poster said only API changes were reasons to bump the
> > major, right?
>
> Yes. He meant *binary* API changes, though, ie, anything that would
> break extant executables originally linked to the prior version of the
> shared library.

Just for clarification --- don't most/all our releases make a binary
change that needs are compile? Actually, you are saying a recompile of
the client, right? Yes, we do those rarely, and in 7.3, for the NOTIFY
structure change. What we do often is want old binaries to use our new
libraries. That isn't a major bump requirement, right?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> Just for clarification --- don't most/all our releases make a binary
> change that needs are compile?

No, they usually don't. Adding new routines, for example, does not
break existing clients.

> What we do often is want old binaries to use our new
> libraries. That isn't a major bump requirement, right?

Exactly. A major bump means that they can't use it.

regards, tom lane

Making something binary incompatible IS an API change! In the case in
question an externally visible structure definition was changed -
clearly a change of API...

Bruce Momjian writes:
> Tom Lane wrote:
> > Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us> writes:
> > > So if a recompile fixes it, increment minor, else major.
> >
> > Wrong --- if you need a recompile then it's not binary-compatible, so
> > it should be a major version bump.
>
> But the previous poster said only API changes were reasons to bump the
> major, right?

Guys, can I take this chance to summarise the thread and when the
major and minor versions should be updated, perhaps could be added to
the developers FAQ if everyone is in agreement?

Major Version
=============

The major version number should be updated whenever the source of the
library changes to make it binary incompatible. Such changes include,
but limited to:

1. Removing a public function or structure (or typedef, enum, ...)

2. Modifying a public functions arguments.

3. Removing a field from a public structure.

3. Adding a field to a public structure, unless steps have been
previously taken to shield users from such a change, for example by
such structures only ever being allocated/instantiated by a library
function which would give the new field a suitable default value.

Adding a new function would NOT force an increase in the major version
number. When the major version is increased all applications which
link to the library MUST be recompiled - this is not desirable. When
the major version is updated the minor version gets reset.

Minor Version
=============

The minor version number should be updated whenever the functionality
of the library has changed, typically and change in source code
between releases would mean an increase in the minor version number so
long as it does not require a major version increase.

Thanks, Lee.

Guys,

Some further comments on bumbing the major version number which aren't
so cut-n-dry...

Lee Kindness writes:
> The major version number should be updated whenever the source of the
> library changes to make it binary incompatible. Such changes include,
> but limited to:
>
> 1. Removing a public function or structure (or typedef, enum, ...)
>
> 2. Modifying a public functions arguments.
>
> 3. Removing a field from a public structure.
>
> 3. Adding a field to a public structure, unless steps have been
> previously taken to shield users from such a change, for example by
> such structures only ever being allocated/instantiated by a library
> function which would give the new field a suitable default value.

For #2 steps could be taken to maintain binary compatibility across
minor PostgreSQL releases (e.g. the 7.2 series, the 7.3 series, the
7.4/8.0 series). Consider the following function

void print_stuff(int arg1, int arg2)
{
printf("stuff: %d %d\n", arg1, arg2);
}

If we wanted to add a third argument:

void print_stuff(int arg1, int arg2, int arg3)
{
printf("stuff: %d %d %d\n", arg1, arg2, arg3);
}

Then doing it like this:

void print_stuff2(int arg1, int arg2, int arg3)
{
printf("stuff: %d %d %d\n", arg1, arg2, arg3);
}

void print_stuff(int arg1, int arg2)
{
print_stuff(arg1, arg2, 0);
}

would maintain binary compatibility. Obviously this would add a fair
bit of cruft if used extensively, but considering the changes between
minor versions would probably be worthwhile to avoid bumping library
major version. Naturally in the next major version print_stuff() would
assume the functionality and arguments of print_stuff2().

Lee.

Lee Kindness wrote:
> Making something binary incompatible IS an API change! In the case in
> question an externally visible structure definition was changed -
> clearly a change of API...

My point was that I thought it was a source-level API change that
required a major bump. I now see it can be a binary change too, so
anything that requires a client recompile is a major bump.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian writes:

> > No, the run-time linker only looks at the major version.
>
> Then what value is there to incrementing the minor number?

For those platforms that have an ldconfig program, the ldconfig will
update the symlinks to the shared library based on the minor version
number. So if you have installed somewhere

libpq.so.2.2
libpq.so.2.3

then ldconfig will create a symlink

libpq.so.2 -> libpq.so.2.3

because that is the newest/best version. Note that "libpq.so.2" is the
file name that the dynamic loader will actually look for.

Other platforms probably have a mechanism that comes out to the same
effect.

--
Peter Eisentraut peter_e(at)gmx(dot)net

OK, I have added this to tools/RELEASE_CHANGES. New file attached.

---------------------------------------------------------------------------

Lee Kindness wrote:
> Guys, can I take this chance to summarise the thread and when the
> major and minor versions should be updated, perhaps could be added to
> the developers FAQ if everyone is in agreement?
>
> Major Version
> =============
>
> The major version number should be updated whenever the source of the
> library changes to make it binary incompatible. Such changes include,
> but limited to:
>
> 1. Removing a public function or structure (or typedef, enum, ...)
>
> 2. Modifying a public functions arguments.
>
> 3. Removing a field from a public structure.
>
> 3. Adding a field to a public structure, unless steps have been
> previously taken to shield users from such a change, for example by
> such structures only ever being allocated/instantiated by a library
> function which would give the new field a suitable default value.
>
> Adding a new function would NOT force an increase in the major version
> number. When the major version is increased all applications which
> link to the library MUST be recompiled - this is not desirable. When
> the major version is updated the minor version gets reset.
>
> Minor Version
> =============
>
> The minor version number should be updated whenever the functionality
> of the library has changed, typically and change in source code
> between releases would mean an increase in the minor version number so
> long as it does not require a major version increase.
>
> Thanks, Lee.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/users-lounge/docs/faq.html
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

I have added this info too.

---------------------------------------------------------------------------

Lee Kindness wrote:
> Guys,
>
> Some further comments on bumbing the major version number which aren't
> so cut-n-dry...
>
> Lee Kindness writes:
> > The major version number should be updated whenever the source of the
> > library changes to make it binary incompatible. Such changes include,
> > but limited to:
> >
> > 1. Removing a public function or structure (or typedef, enum, ...)
> >
> > 2. Modifying a public functions arguments.
> >
> > 3. Removing a field from a public structure.
> >
> > 3. Adding a field to a public structure, unless steps have been
> > previously taken to shield users from such a change, for example by
> > such structures only ever being allocated/instantiated by a library
> > function which would give the new field a suitable default value.
>
> For #2 steps could be taken to maintain binary compatibility across
> minor PostgreSQL releases (e.g. the 7.2 series, the 7.3 series, the
> 7.4/8.0 series). Consider the following function
>
> void print_stuff(int arg1, int arg2)
> {
> printf("stuff: %d %d\n", arg1, arg2);
> }
>
> If we wanted to add a third argument:
>
> void print_stuff(int arg1, int arg2, int arg3)
> {
> printf("stuff: %d %d %d\n", arg1, arg2, arg3);
> }
>
> Then doing it like this:
>
> void print_stuff2(int arg1, int arg2, int arg3)
> {
> printf("stuff: %d %d %d\n", arg1, arg2, arg3);
> }
>
> void print_stuff(int arg1, int arg2)
> {
> print_stuff(arg1, arg2, 0);
> }
>
> would maintain binary compatibility. Obviously this would add a fair
> bit of cruft if used extensively, but considering the changes between
> minor versions would probably be worthwhile to avoid bumping library
> major version. Naturally in the next major version print_stuff() would
> assume the functionality and arguments of print_stuff2().
>
> Lee.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073