ALTER EXTENSION UPGRADE patch v1

Hi,

Please find attached the patch to support ALTER EXTENSION UPGRADE,
following exchanges made on this list — we can't really speak about
decisions here, apparently, until commit is done :)

The documentation is available online for easy browsing here:

http://pgsql.tapoueh.org/extensions/doc/html/extend-extension.html

The things we might want to discuss are, AFAIUI:

- support for upgrading from pre-9.1 extension, or to get from a bunch
of related pl code to a proper extension in your database

I've done the code for that to work, the idea is pretty simple and
relies on the following points. Please note that the attached patch
only implements support for upgrading some contrib modules from
pre-9.1, that's because I'm willing to get some approvement before
continuing the effort — I foresee no road blocks here.

- CREATE WRAPPER EXTENSION

If you want to upgrade an extension, it first has to exists as an
extension. When you're wanting to switch to extension, you need a
way to bootstrap your extension object, because you want an entry in
the catalogs and you don't want to run the script: it has already
been run before you decided this would make for an extension.

This command allows you to register an extension in the catalogs,
bypassing any script and forcing the version to NULL.

I'm not sold on the 'WRAPPER' keyword here, I just used an existing
one in the grammar that looked like a good candidate. This was the
best I could find, but IANANS (I am not a native speaker).

- ALTER EXTENSION ... UPGRADE

This allows you to upgrade an existing extension, wrapper included.

The extension author needs to prepare the upgrade scripts, and to
setup in the control file which to use. The new extension's version
is static as the upgrade files you distribute depend on this (think
packaging here) and so you pick the upgrade script file to execute
depending only on the currently installed version.

This picking is made by regular expression matching (using the code
backing the ~(text, text) operator), and supports null as a special
case. The first match wins, other candidates are bypassed.

- CREATE EXTENSION ... WITH SYSID ...

Thinking about upgrades, I added support for that too. This might be
necessary for pg_upgrade, but I'm not so sure. It was an experiment
but I though we might as well talk about it before I remove it from
the patch :)

- ALTER OBJECT ... SET EXTENSION ...

This implements the magic behind "upgrading from null", as it allows
the users to add objects to an existing extension. Internally, this
will only record a INTERNAL depedency between the object and the
extension.

This is mainly for use in upgrade scripts when you support pre-9.1
extension upgrade, or for internal PL code that grows into an
extension after the fact.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Tue, Jan 04, 2011 at 12:31:55PM +0100, Dimitri Fontaine wrote:
> Hi,
>
> Please find attached the patch to support ALTER EXTENSION UPGRADE,

Do you plan to have

ALTER EXTENSION ... UPGRADE TO VERSION ...

, or the more general,

ALTER EXTENSION ... ALTER VERSION TO ... ?

I get that this might not be a 9.1 feature, but it's sure to be one
people who need to deploy software want. A lot.

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David Fetter <david(at)fetter(dot)org> writes:
> Do you plan to have
>
> ALTER EXTENSION ... UPGRADE TO VERSION ...
>
> , or the more general,
>
> ALTER EXTENSION ... ALTER VERSION TO ... ?

Well why not, but I'm not sure I understand what you have in mind here.
I don't exactly see how to install more than one version on the server,
at the filesystem level, with the current infrastructure.

The way debian solves it for multi-versions packages, such as PostgreSQL
for example, is to add the (major) version number in the package name,
so that you have independent packages such as:
postgresql-8.3
postgresql-8.4
postgresql-9.0

Are you thinking about doing something similar with extensions?

The only other way I'm able to think about your request would be for the
command to simply ERROR out when the premises given by the users are not
matching the reality of what's installed both in the catalogs and on the
file system. If that's what you're talking about, we still can make it
material for 9.1, I believe.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Tue, Jan 04, 2011 at 08:31:19PM +0100, Dimitri Fontaine wrote:
> David Fetter <david(at)fetter(dot)org> writes:
> > Do you plan to have
> >
> > ALTER EXTENSION ... UPGRADE TO VERSION ...
> >
> > , or the more general,
> >
> > ALTER EXTENSION ... ALTER VERSION TO ... ?
>
> Well why not, but I'm not sure I understand what you have in mind here.

One could imagine that an extension was updated more quickly than
PostgreSQL major versions come out, or at least not at the exact same
time.

> I don't exactly see how to install more than one version on the
> server, at the filesystem level, with the current infrastructure.

It's probably not worth worrying about the multiple version issue for
9.1, but it probably *is* for 9.2.

> The way debian solves it for multi-versions packages, such as PostgreSQL
> for example, is to add the (major) version number in the package name,
> so that you have independent packages such as:
> postgresql-8.3
> postgresql-8.4
> postgresql-9.0
>
> Are you thinking about doing something similar with extensions?

Vaguely. See above.

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David Fetter <david(at)fetter(dot)org> writes:
> One could imagine that an extension was updated more quickly than
> PostgreSQL major versions come out, or at least not at the exact same
> time.

Sure, but I don't see what your proposed syntax is giving us here.
Currently the new version you're upgraded to is the one installed on the
file system, and whose .so has already been upgraded, and that's it.

I still need a better grasp at what your proposal is about to comment
and integrate, I fear.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Wed, Jan 05, 2011 at 10:19:23AM +0100, Dimitri Fontaine wrote:
> David Fetter <david(at)fetter(dot)org> writes:
> > One could imagine that an extension was updated more quickly than
> > PostgreSQL major versions come out, or at least not at the exact same
> > time.
>
> Sure, but I don't see what your proposed syntax is giving us here.

The syntax by itself does nothing, but the underlying capability gives
users:

- The ability to have versions of software on different databases on
the same system.

- The ability to do deterministic upgrades, rather than just, "upgrade
me to the latest, which may be buggy and/or slow things down to
avoid a problem I know I don't have."

> Currently the new version you're upgraded to is the one installed on
> the file system, and whose .so has already been upgraded, and that's
> it.
>
> I still need a better grasp at what your proposal is about to
> comment and integrate, I fear.

I am not saying that this is a show-stopper. I *am* saying that
multiple concurrent versions and deterministic upgrades are common
enough requests that you shouldn't do things that would prevent those
in the future.

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

David Fetter <david(at)fetter(dot)org> writes:
> The syntax by itself does nothing, but the underlying capability gives
> users:

Ok, now I understand that the syntax you proposed was a shortcut for an
I-want-it-all request :)

> - The ability to have versions of software on different databases on
> the same system.
>
> - The ability to do deterministic upgrades, rather than just, "upgrade
> me to the latest, which may be buggy and/or slow things down to
> avoid a problem I know I don't have."

Both depends on a filesystem organization rework.

> I am not saying that this is a show-stopper. I *am* saying that
> multiple concurrent versions and deterministic upgrades are common
> enough requests that you shouldn't do things that would prevent those
> in the future.

Would it be useful to have the syntax support in 9.1 already, but which
would only check that the asked-of new version (and current version) are
the one currently available (and installed), and ERROR out otherwise?

I think that syntax-and-check is still doable for 9.1, if there's a will
to get there.

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support