Runtime SHAREDIR for testing CREATE EXTENSION

I'm trying to understand what options I have to test "CREATE EXTENSION"
w/out installing the extension files in their final destination.

Could not find a way to set SHAREDIR from within psql, nor a way
to specify it in initdb call. Am I missing it something ?

PS: please include my address in replies as I have mail delivery
disabled from the list. Thank you.

--strk;

,------o-.
| __/ | Delivering high quality PostGIS 2.0 !
| / 2.0 | http://strk.keybit.net
`-o------'

Sandro Santilli <strk(at)keybit(dot)net> writes:
> I'm trying to understand what options I have to test "CREATE EXTENSION"
> w/out installing the extension files in their final destination.

There aren't any. Generally speaking, if you want to be testing an
extension, you should be doing it in a test installation anyway;
so I don't see what's the objection to sticking the files into the
installation's real SHAREDIR.

regards, tom lane

On Tue, Feb 21, 2012 at 10:21:17AM -0500, Tom Lane wrote:
> Sandro Santilli <strk(at)keybit(dot)net> writes:
> > I'm trying to understand what options I have to test "CREATE EXTENSION"
> > w/out installing the extension files in their final destination.
>
> There aren't any. Generally speaking, if you want to be testing an
> extension, you should be doing it in a test installation anyway;
> so I don't see what's the objection to sticking the files into the
> installation's real SHAREDIR.

We do a staged installation of the extension library and scripts
in an ad-hoc dir which does not match the built-in SHAREDIR of PostgreSQL.

It works fine for testing the extension directly sourcing the scripts.
It's only "CREATE EXTENSION" that doesn't play nicely with that setup.

--strk;

Sandro Santilli <strk(at)keybit(dot)net> writes:
> On Tue, Feb 21, 2012 at 10:21:17AM -0500, Tom Lane wrote:
>> Sandro Santilli <strk(at)keybit(dot)net> writes:
>> > I'm trying to understand what options I have to test "CREATE EXTENSION"
>> > w/out installing the extension files in their final destination.
>>
>> There aren't any. Generally speaking, if you want to be testing an
>> extension, you should be doing it in a test installation anyway;
>> so I don't see what's the objection to sticking the files into the
>> installation's real SHAREDIR.
>
> We do a staged installation of the extension library and scripts
> in an ad-hoc dir which does not match the built-in SHAREDIR of PostgreSQL.
>
> It works fine for testing the extension directly sourcing the scripts.
> It's only "CREATE EXTENSION" that doesn't play nicely with that setup.

Please see the "inline extension" thread where answers to your problem
have been discussed.

https://commitfest.postgresql.org/action/patch_view?id=746

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Tue, Feb 21, 2012 at 10:34:42PM +0100, Dimitri Fontaine wrote:
> Sandro Santilli <strk(at)keybit(dot)net> writes:
> > On Tue, Feb 21, 2012 at 10:21:17AM -0500, Tom Lane wrote:
> >> Sandro Santilli <strk(at)keybit(dot)net> writes:
> >> > I'm trying to understand what options I have to test "CREATE EXTENSION"
> >> > w/out installing the extension files in their final destination.
> >>
> >> There aren't any. Generally speaking, if you want to be testing an
> >> extension, you should be doing it in a test installation anyway;
> >> so I don't see what's the objection to sticking the files into the
> >> installation's real SHAREDIR.
> >
> > We do a staged installation of the extension library and scripts
> > in an ad-hoc dir which does not match the built-in SHAREDIR of PostgreSQL.
> >
> > It works fine for testing the extension directly sourcing the scripts.
> > It's only "CREATE EXTENSION" that doesn't play nicely with that setup.
>
> Please see the "inline extension" thread where answers to your problem
> have been discussed.
>
> https://commitfest.postgresql.org/action/patch_view?id=746

I'm not really looking for "inline extensions".
I do want to install the extension objects somewhere, just NOT
in the PostgreSQL builtin SHAREDIR but in an arbitrary staging
directory to use for QA the extension before distribution.

--strk;

,------o-.
| __/ | Delivering high quality PostGIS 2.0 !
| / 2.0 | http://strk.keybit.net
`-o------'

On Tue, Feb 21, 2012 at 1:34 PM, Dimitri Fontaine
<dimitri(at)2ndquadrant(dot)fr> wrote:
> Sandro Santilli <strk(at)keybit(dot)net> writes:
> Please see the "inline extension" thread where answers to your problem
> have been discussed.

I'm pretty sure Sandro is hacking PostGIS, so inline extensions are of
no help here.

Can you tell us why alternative approaches that don't require adding a
knob (such as copying/symlinking of compiled artifacts) is such a huge
pain for you?

--
fdr

Re: Sandro Santilli 2012-02-22 <20120222101656(dot)GB6125(at)gnash>
> I'm not really looking for "inline extensions".
> I do want to install the extension objects somewhere, just NOT
> in the PostgreSQL builtin SHAREDIR but in an arbitrary staging
> directory to use for QA the extension before distribution.

We have the same problem with testing extensions at build-time in the
Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
only writable by root, while the build is running as buildd user, so
there is no way to do "create extension whatimbuildingrightnow" to be
able to run regression tests, even if this is a cluster I have just
created with initdb.

Christoph
--
cb(at)df7cb(dot)de | http://www.df7cb.de/

Daniel Farina <daniel(at)heroku(dot)com> writes:
> I'm pretty sure Sandro is hacking PostGIS, so inline extensions are of
> no help here.

Sorry about the sloppy mention, one of the things under discussion (but
out of scope for 9.2) in the inline extension thread is about where to
store the shared objects and extension files so that we have a real
per-database objects here, rather than per major version deployments
that happen to be able to impact several clusters and databases in
different and subtle ways.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Fri, Feb 24, 2012 at 09:59:12AM +0100, Christoph Berg wrote:
> Re: Sandro Santilli 2012-02-22 <20120222101656(dot)GB6125(at)gnash>
> > I'm not really looking for "inline extensions".
> > I do want to install the extension objects somewhere, just NOT
> > in the PostgreSQL builtin SHAREDIR but in an arbitrary staging
> > directory to use for QA the extension before distribution.
>
> We have the same problem with testing extensions at build-time in the
> Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
> only writable by root, while the build is running as buildd user, so
> there is no way to do "create extension whatimbuildingrightnow" to be
> able to run regression tests, even if this is a cluster I have just
> created with initdb.

Exactly. We don't initdb with PostGIS regression testing framework
but I've considered doing it for this specific case and it stroke me
that even then we couldn't control SHAREDIR.

--strk;

,------o-.
| __/ | Delivering high quality PostGIS 2.0 !
| / 2.0 | http://strk.keybit.net - http://vizzuality.com
`-o------'

On Thu, Feb 23, 2012 at 06:53:05PM -0800, Daniel Farina wrote:
> On Tue, Feb 21, 2012 at 1:34 PM, Dimitri Fontaine
> <dimitri(at)2ndquadrant(dot)fr> wrote:
> > Sandro Santilli <strk(at)keybit(dot)net> writes:
> > Please see the "inline extension" thread where answers to your problem
> > have been discussed.
>
> I'm pretty sure Sandro is hacking PostGIS, so inline extensions are of
> no help here.
>
> Can you tell us why alternative approaches that don't require adding a
> knob (such as copying/symlinking of compiled artifacts) is such a huge
> pain for you?

It's not specifically a pain for me, but it breaks the
"tests can be run in a staged environment" idiom.

See Christoph Berg mail, that's exactly the same issue.

--strk;

,------o-.
| __/ | Delivering high quality PostGIS 2.0 !
| / 2.0 | http://strk.keybit.net - http://vizzuality.com
`-o------'

Christoph Berg <cb(at)df7cb(dot)de> writes:
> Re: Sandro Santilli 2012-02-22 <20120222101656(dot)GB6125(at)gnash>
>> I'm not really looking for "inline extensions".
>> I do want to install the extension objects somewhere, just NOT
>> in the PostgreSQL builtin SHAREDIR but in an arbitrary staging
>> directory to use for QA the extension before distribution.

> We have the same problem with testing extensions at build-time in the
> Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
> only writable by root, while the build is running as buildd user, so
> there is no way to do "create extension whatimbuildingrightnow" to be
> able to run regression tests, even if this is a cluster I have just
> created with initdb.

This seems like nonsense. If the build process has installed the
software, you surely have got permissions to write in that directory.
If you haven't installed the software, you need to do testing in a
temporary installation per "make check", and we are able to test
extensions that way too.

regards, tom lane

On fre, 2012-02-24 at 11:53 -0500, Tom Lane wrote:
> > We have the same problem with testing extensions at build-time in
> the
> > Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
> > only writable by root, while the build is running as buildd user, so
> > there is no way to do "create extension whatimbuildingrightnow" to
> be
> > able to run regression tests, even if this is a cluster I have just
> > created with initdb.
>
> This seems like nonsense. If the build process has installed the
> software, you surely have got permissions to write in that directory.

The build process just installs the software in a fake root where it
will be wrapped up by the packaging software.

> If you haven't installed the software, you need to do testing in a
> temporary installation per "make check", and we are able to test
> extensions that way too.
>
That looks like the right answer.

On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote:
> We don't initdb with PostGIS regression testing framework
> but I've considered doing it for this specific case and it stroke me
> that even then we couldn't control SHAREDIR.

I would always create a new instance using initdb for test runs. It's
easy and avoids all these problems.

On Fri, Feb 24, 2012 at 10:21 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote:
>> We don't initdb with PostGIS regression testing framework
>> but I've considered doing it for this specific case and it stroke me
>> that even then we couldn't control SHAREDIR.
>
> I would always create a new instance using initdb for test runs.  It's
> easy and avoids all these problems.

Having been in this position once before in a different but similar
situation, there's one big caveat: initdb is *really* slow, so it is
really painful for people who write Postgres-linked code that is
compiled separately, whereby the initdb binary is stable. The
frustration is not unlike when I have to run ./configure, except I had
to do it all the time, every time.

A ccache-like solution has worked well for at least one person I know,
and wasn't hard to implement(?) for personal use.

But by hook or crook, a fresh database is the "right" thing for sure.

--
fdr

On Fri, Feb 24, 2012 at 3:31 PM, Daniel Farina <daniel(at)heroku(dot)com> wrote:
> Having been in this position once before in a different but similar
> situation, there's one big caveat: initdb is *really* slow, so it is
> really painful for people who write Postgres-linked code that is
> compiled separately, whereby the initdb binary is stable.  The
> frustration is not unlike when I have to run ./configure, except I had
> to do it all the time, every time.
>
> A ccache-like solution has worked well for at least one person I know,
> and wasn't hard to implement(?) for personal use.

Being the person in question: I don't have the full context so I'm not
sure how useful this will be, but here's hoping it's not just line
noise. I'm probably not at liberty to share the original code, but it
was a fairly trivial shell script that did more or less the following:

1. Capture all arguments (to pass to initdb itself) except the path,
and figure out the path to the cluster (via looking for
-D/--pgdata/$PGDATA)
2. Create a hash of the concatenation of the arguments (again, sans
cluster path)
3. Create a "template" path based on a local cache directory and the
argument hash
4. a) If the directory with this path did not exist, run initdb with
-D set to this and the rest of the arguments as captured; then copy to
*actual* path
b) If it does exist, that's because you've run initdb with these
arguments before: just copy that to the cluster path you captured in
step (1) instead of starting fresh

This was not terribly robust (e.g., I remember that Dan had pointed
out several shortcomings), but none of these were practical issues for
me, and I never had a problem. There is some more work around
capturing the cluster path in the initdb output and mangling it back
from the template version into the original (so the output of the
cached initdb looks like output of a real initdb), but that's not a
big deal. The approach could probably be made solid enough for
general-purpose use.

On 02/24/2012 06:31 PM, Daniel Farina wrote:
> On Fri, Feb 24, 2012 at 10:21 AM, Peter Eisentraut<peter_e(at)gmx(dot)net> wrote:
>> On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote:
>>> We don't initdb with PostGIS regression testing framework
>>> but I've considered doing it for this specific case and it stroke me
>>> that even then we couldn't control SHAREDIR.
>> I would always create a new instance using initdb for test runs. It's
>> easy and avoids all these problems.
> Having been in this position once before in a different but similar
> situation, there's one big caveat: initdb is *really* slow, so it is
> really painful for people who write Postgres-linked code that is
> compiled separately, whereby the initdb binary is stable. The
> frustration is not unlike when I have to run ./configure, except I had
> to do it all the time, every time.

Really? Here's what I just got on a severely under-resourced SL6 VM:

[andrew(at)xena ~]$ time /usr/pgsql-9.1/bin/initdb -A ident foo > /dev/null

real 0m1.441s
user 0m1.131s
sys 0m0.309s

1.5s doesn't seem terribly slow.

cheers

andrew

On Fri, Feb 24, 2012 at 6:31 PM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
> Really? Here's what I just got on a severely under-resourced SL6 VM:
>
> 1.5s doesn't seem terribly slow.

You are right. Come to think of it, I do seem to recall that initdb
got some speed improvements; these were in 8.3 days, I think.

Here are my numbers for comparison, in some various forms:

env time /usr/lib/postgresql/9.1/bin/initdb /tmp/foo
0.94user 0.28system 0:01.87elapsed 64%CPU (0avgtext+0avgdata 79920maxresident)k
14288inputs+85144outputs (36major+66622minor)pagefaults 0swaps

env time eatmydata /usr/lib/postgresql/9.1/bin/initdb /tmp/foo
0.90user 0.27system 0:01.26elapsed 92%CPU (0avgtext+0avgdata 79984maxresident)k
856inputs+85144outputs (0major+67202minor)pagefaults 0swaps

env time /usr/lib/postgresql/9.1/bin/initdb /dev/shm/blah
0.90user 0.19system 0:01.15elapsed 95%CPU (0avgtext+0avgdata 79984maxresident)k
0inputs+0outputs (0major+67192minor)pagefaults 0swaps

Retracted.

--
fdr

Re: Peter Eisentraut 2012-02-24 <1330107599(dot)32452(dot)15(dot)camel(at)vanquo(dot)pezone(dot)net>
> On fre, 2012-02-24 at 11:53 -0500, Tom Lane wrote:
> > > We have the same problem with testing extensions at build-time in
> > the
> > > Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
> > > only writable by root, while the build is running as buildd user, so
> > > there is no way to do "create extension whatimbuildingrightnow" to
> > be
> > > able to run regression tests, even if this is a cluster I have just
> > > created with initdb.
> >
> > This seems like nonsense. If the build process has installed the
> > software, you surely have got permissions to write in that directory.
>
> The build process just installs the software in a fake root where it
> will be wrapped up by the packaging software.
>
> > If you haven't installed the software, you need to do testing in a
> > temporary installation per "make check", and we are able to test
> > extensions that way too.
> >
> That looks like the right answer.

Well, I'm trying to invoke the extension's "make check" target at
extension build time. I do have a temporary installation I own
somehwere in my $HOME, but that is still trying to find extensions in
/usr/share/postgresql/9.1/extension/*.control, because I am using the
system's postgresql version. The build process is not running as root,
so I cannot do an install of the extension to its final location.
Still it would be nice to run regression tests. All that seems to be
missing is the ability to put

extension_control_path = /home/buildd/tmp/extension

into the postgresql.conf of the temporary PG installation, or some
other way like "CREATE EXTENSION foobar WITH CONTROL '/home/buildd/...'.

Christoph
--
cb(at)df7cb(dot)de | http://www.df7cb.de/

On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote:
> Well, I'm trying to invoke the extension's "make check" target at
> extension build time. I do have a temporary installation I own
> somehwere in my $HOME, but that is still trying to find extensions in
> /usr/share/postgresql/9.1/extension/*.control, because I am using the
> system's postgresql version. The build process is not running as root,
> so I cannot do an install of the extension to its final location.
> Still it would be nice to run regression tests. All that seems to be
> missing is the ability to put
>
> extension_control_path = /home/buildd/tmp/extension
>
> into the postgresql.conf of the temporary PG installation, or some
> other way like "CREATE EXTENSION foobar WITH CONTROL
> '/home/buildd/...'.

Yeah, of course, the extension path is not related to the data
directory. So we do need some kind of path setting, just like
dynamic_library_path.

On Sun, Feb 26, 2012 at 10:36 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote:
>> Well, I'm trying to invoke the extension's "make check" target at
>> extension build time. I do have a temporary installation I own
>> somehwere in my $HOME, but that is still trying to find extensions in
>> /usr/share/postgresql/9.1/extension/*.control, because I am using the
>> system's postgresql version. The build process is not running as root,
>> so I cannot do an install of the extension to its final location.
>> Still it would be nice to run regression tests. All that seems to be
>> missing is the ability to put
>>
>> extension_control_path = /home/buildd/tmp/extension
>>
>> into the postgresql.conf of the temporary PG installation, or some
>> other way like "CREATE EXTENSION foobar WITH CONTROL
>> '/home/buildd/...'.
>
> Yeah, of course, the extension path is not related to the data
> directory.  So we do need some kind of path setting, just like
> dynamic_library_path.

That logic seems sound to me, so +1.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Feb 24, 2012 at 08:21:05PM +0200, Peter Eisentraut wrote:
> On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote:
> > We don't initdb with PostGIS regression testing framework
> > but I've considered doing it for this specific case and it stroke me
> > that even then we couldn't control SHAREDIR.
>
> I would always create a new instance using initdb for test runs. It's
> easy and avoids all these problems.

Doesn't avoid the SHAREDIR problem, that's what I'm saying.
SHAREDIR is a compile-time setting in PostgreSQL.

--strk;

On Sun, Feb 26, 2012 at 09:50:04PM -0500, Robert Haas wrote:
> On Sun, Feb 26, 2012 at 10:36 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> > On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote:
> >> Well, I'm trying to invoke the extension's "make check" target at
> >> extension build time. I do have a temporary installation I own
> >> somehwere in my $HOME, but that is still trying to find extensions in
> >> /usr/share/postgresql/9.1/extension/*.control, because I am using the
> >> system's postgresql version. The build process is not running as root,
> >> so I cannot do an install of the extension to its final location.
> >> Still it would be nice to run regression tests. All that seems to be
> >> missing is the ability to put
> >>
> >> extension_control_path = /home/buildd/tmp/extension
> >>
> >> into the postgresql.conf of the temporary PG installation, or some
> >> other way like "CREATE EXTENSION foobar WITH CONTROL
> >> '/home/buildd/...'.
> >
> > Yeah, of course, the extension path is not related to the data
> > directory.  So we do need some kind of path setting, just like
> > dynamic_library_path.
>
> That logic seems sound to me, so +1.

+1 here as well. I may attempt to produce a patch if this gets consensus.

--strk;

,------o-.
| __/ | Delivering high quality PostGIS 2.0 !
| / 2.0 | http://strk.keybit.net - http://vizzuality.com
`-o------'

On Sun, Feb 26, 2012 at 7:36 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote:
>> Well, I'm trying to invoke the extension's "make check" target at
>> extension build time. I do have a temporary installation I own
>> somehwere in my $HOME, but that is still trying to find extensions in
>> /usr/share/postgresql/9.1/extension/*.control, because I am using the
>> system's postgresql version. The build process is not running as root,
>> so I cannot do an install of the extension to its final location.
>> Still it would be nice to run regression tests. All that seems to be
>> missing is the ability to put
>>
>> extension_control_path = /home/buildd/tmp/extension
>>
>> into the postgresql.conf of the temporary PG installation, or some
>> other way like "CREATE EXTENSION foobar WITH CONTROL
>> '/home/buildd/...'.
>
> Yeah, of course, the extension path is not related to the data
> directory.  So we do need some kind of path setting, just like
> dynamic_library_path.

A thought that may or may not be related:

I'd really like to support libraries (C or otherwise) of multiple
versions at the same time, when the underlying library permits. The
critical use case is contribs that iterate on their storage format,
which right now is really painful. While perhaps contribs should use
a version byte or something and be backwards-compatible forever, to
some extent that is clearly impossible and sometimes undesirable for
contribs that emit small on-disk representations, and would rather
use, say, the type system to disambiguate what operators fit with what
data.

I've been struggling to think of a more graceful way to load the
correct version of extra dependencies (including non-PG intrinsic
things like libgeos or libproj) when new database replicas are
created, as well as helping people with upgrade issues. Is this
tunable possibly related to a solution for this?

--
fdr

On tis, 2012-02-28 at 11:00 -0800, Daniel Farina wrote:
> I'd really like to support libraries (C or otherwise) of multiple
> versions at the same time, when the underlying library permits.

What's preventing you from doing that now? You need to name all the
symbols differently, of course.

On Fri, Mar 2, 2012 at 10:37 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On tis, 2012-02-28 at 11:00 -0800, Daniel Farina wrote:
>> I'd really like to support libraries (C or otherwise) of multiple
>> versions at the same time, when the underlying library permits.
>
> What's preventing you from doing that now?  You need to name all the
> symbols differently, of course.

That's the problem: not really practical in a wider ecosystem of C
libraries, especially if the library produces multiple versions. (Or,
not practical unless someone writes some credible
symbol-version-mangling-magic)

But is it unsurmountable? -- dlsym returns a function pointer, and one
would build up the operator table for the version of the extension at
hand, so one might have ltree "version 1.01" and ltree "version 2.3"
fields in the same database.

--
fdr

On Fri, Mar 2, 2012 at 1:53 PM, Daniel Farina <daniel(at)heroku(dot)com> wrote:
> On Fri, Mar 2, 2012 at 10:37 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
>> On tis, 2012-02-28 at 11:00 -0800, Daniel Farina wrote:
>>> I'd really like to support libraries (C or otherwise) of multiple
>>> versions at the same time, when the underlying library permits.
>>
>> What's preventing you from doing that now?  You need to name all the
>> symbols differently, of course.
>
> That's the problem: not really practical in a wider ecosystem of C
> libraries, especially if the library produces multiple versions.  (Or,
> not practical unless someone writes some credible
> symbol-version-mangling-magic)
>
> But is it unsurmountable? -- dlsym returns a function pointer, and one
> would build up the operator table for the version of the extension at
> hand, so one might have ltree "version 1.01" and ltree "version 2.3"
> fields in the same database.

That might be possible, but it seems largely unrelated to the topic of
this thread, unless I am missing something.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Mar 2, 2012 at 1:50 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> But is it unsurmountable? -- dlsym returns a function pointer, and one
>> would build up the operator table for the version of the extension at
>> hand, so one might have ltree "version 1.01" and ltree "version 2.3"
>> fields in the same database.
>
> That might be possible, but it seems largely unrelated to the topic of
> this thread, unless I am missing something.

It might be, and might not be, because if we start getting into where
to put shared objects for functionality linked to the database (rather
just letting the dynamic linker do "whatever") it could crash head-on
into data field versioning and replica issues. If someone thinks
about it for a few minutes and says "nope, orthogonal," I'm without
objection, but I did want to raise this spectre to other minds because
it seems to be on the edge of that maelstrom.

I also look at the inability for pg_upgrade to help when certain
contribs are used between versions and can only say "phew, I'm glad
that doesn't affect most people" (a sad flip side: people aren't using
contribs and database extension as much as I like, but things are
moving towards more of that and not less) because then the expected
duration of the upgrade process becomes a very ugly compatibility
matrix that nobody wants to see...somewhat dissatisfying for a service
provider. Yet, formats can and will change requiring a lot of
gradual I/O, and a version byte is not always practical, so the best
option I can think of is to support multiple versions of formats,
operators and types at the same time, and then gradually rewrite
things.

I'm not intending to open discussion about any of that right now
(unless someone else is interested and wants to start a new thread),
but I wanted to provide insight as to to why I'm especially
preoccupied about this.

--
fdr