Re: Avoiding Application Re-test

Simon Riggs wrote:
> Tom's recent changes to allow hash distinct (yay!) prompted something
> that I'd thought about previously.
>
> Subtle changes in the output of queries can force an application retest,
> which then can slow down or prevent an upgrade to the latest release. We
> always assume the upgrade itself is the problem, but the biggest barrier
> I see is the cost and delay involved in upgrading the application.
>
> We could invent a new parameter called enable_sort_distinct, but thats
> way too specific and horrible.
>
> What I would like is a parameter called sql_compatibility which has
> settings such as 8.3, 8.4 etc.. By default it would have the value 8.4,
> but for people that want to upgrade *without* retesting their
> application, they could set it to 8.3.
>
> Every time we introduce a feature that changes output, we just put an if
> test in saying sql_compatibility = X, (the release we added feature).
>
> Straightforward, futureproof. Cool.
>
> Not foolproof, but still worth it. This would allow many users to
> upgrade to 8.4 for new features, yet without changing apps.

Won't there normally be a number of changes that *cannot* be covered by
such a parameter, without a whole lot more work in the patch?

If so, people will be led to believe they are safe by setting
"sql_compatibility=8.3", but really they're not, and they will be annoyed.

FWIW, MSSQL has a similar feature. It covers some things and not all.
Personally, I find it annoying because vendors think they don't have to
re-test since they set it lower, but in reality things still broke. But
there are certainly a number of cases where it's useful.

I think it comes down to if there how much more work/code will be needed
in relation to how many things it will actually be possible to cover
with it...

//Magnus

Tom's recent changes to allow hash distinct (yay!) prompted something
that I'd thought about previously.

Subtle changes in the output of queries can force an application retest,
which then can slow down or prevent an upgrade to the latest release. We
always assume the upgrade itself is the problem, but the biggest barrier
I see is the cost and delay involved in upgrading the application.

We could invent a new parameter called enable_sort_distinct, but thats
way too specific and horrible.

What I would like is a parameter called sql_compatibility which has
settings such as 8.3, 8.4 etc.. By default it would have the value 8.4,
but for people that want to upgrade *without* retesting their
application, they could set it to 8.3.

Every time we introduce a feature that changes output, we just put an if
test in saying sql_compatibility = X, (the release we added feature).

Straightforward, futureproof. Cool.

Not foolproof, but still worth it. This would allow many users to
upgrade to 8.4 for new features, yet without changing apps.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support

Simon Riggs wrote:

> What I would like is a parameter called sql_compatibility which has
> settings such as 8.3, 8.4 etc.. By default it would have the value 8.4,
> but for people that want to upgrade *without* retesting their
> application, they could set it to 8.3.

I think down this route lies code readability and maintenability
madness. Right now, the optimizer and executor code is already a very
complex beast. We don't need to make it more difficult to understand
and improve.

Besides, the guys not updating their applications would not be
benefitted by the upgrade if they kept the compatibility with the older
version, so why would they upgrade?

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> Simon Riggs wrote:
>> What I would like is a parameter called sql_compatibility which has
>> settings such as 8.3, 8.4 etc.. By default it would have the value 8.4,
>> but for people that want to upgrade *without* retesting their
>> application, they could set it to 8.3.

> Won't there normally be a number of changes that *cannot* be covered by
> such a parameter, without a whole lot more work in the patch?

Yeah. It's not always possible, and certainly not always easy, to
provide such a behavior. Moreover, an "all or nothing" switch doesn't
seem like it'd be such a big win: presumably, the reason you are updating
to 8.4 is you would like something you can't get in 8.3. So you're
going to have to turn the switch off sooner rather than later, making
the value questionable.

regards, tom lane

Magnus Hagander wrote:
> Simon Riggs wrote:
>>
>> Not foolproof, but still worth it. This would allow many users to
>> upgrade to 8.4 for new features, yet without changing apps.
>
> Won't there normally be a number of changes that *cannot* be covered by
> such a parameter, without a whole lot more work in the patch?

Slightly OT, but just so it's on the record.

I'm of the opinion that whatever packaging system eventually makes it's
way into PG it needs to apply to stuff that's in core at the moment.
That way I can just unload network_addr_types and I no longer have to
worry whether applications might be affected by an incompatible change.

--
Richard Huxton
Archonet Ltd

It would make PostgreSQL too much like Oracle ;) Let's keep PostgreSQL
simple and compact please.
I prefer applications retest when migrating to new PostgreSQL version. In
this case surprises happen then you expect them not in some unforeseen point
of time in the future.
Keeping all this old functionality around will make maintenance and adding
new stuff harder.
It also complicates tracking problems where in addition to db version you
need to find out what version it is supposed to emulate.

On Thu, Aug 7, 2008 at 5:17 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> Tom's recent changes to allow hash distinct (yay!) prompted something
> that I'd thought about previously.
>
> Subtle changes in the output of queries can force an application retest,
> which then can slow down or prevent an upgrade to the latest release. We
> always assume the upgrade itself is the problem, but the biggest barrier
> I see is the cost and delay involved in upgrading the application.
>
> We could invent a new parameter called enable_sort_distinct, but thats
> way too specific and horrible.
>
> What I would like is a parameter called sql_compatibility which has
> settings such as 8.3, 8.4 etc.. By default it would have the value 8.4,
> but for people that want to upgrade *without* retesting their
> application, they could set it to 8.3.
>
> Every time we introduce a feature that changes output, we just put an if
> test in saying sql_compatibility = X, (the release we added feature).
>
> Straightforward, futureproof. Cool.
>
> Not foolproof, but still worth it. This would allow many users to
> upgrade to 8.4 for new features, yet without changing apps.
>
> --
> Simon Riggs www.2ndQuadrant.com
> PostgreSQL Training, Services and Support
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>

simon(at)2ndquadrant(dot)com (Simon Riggs) writes:
> Every time we introduce a feature that changes output, we just put an if
> test in saying sql_compatibility = X, (the release we added feature).
>
> Straightforward, futureproof. Cool.

This is somewhat like the way that some shells try to emulate others;
for instance, zsh tries to emulate "sh" or "ksh" when invoked by those
names.

There are two problems with this:

a) The conspicuous phrase, "try to emulate"

Which leaves open several questions:

- What if we can't?

- What about when we don't want to (e.g. - because the new
version has a new behaviour that we *DO* want)?

- What if some semantic change takes place that we didn't
realize should have been open to the "try to emulate"?
Such as, where there's an ordering change...

b) It requires adding a new, not-previous-existant effort to
classify changes and put in logic to allow controlling whether we
use "legacy logic" or "new logic."

That seems likely to get mighty messy to me, requiring
"blabbering" control code throughout the code base.

I don't think we actually get the "future proofness" that you're
suggesting. It would be nice if we could, but that seems unrealistic.
--
let name="cbbrowne" and tld="linuxdatabases.info" in String.concat "@" [name;tld];;
http://linuxfinances.info/info/spiritual.html
Rules of the Evil Overlord #204. "I will hire an entire squad of blind
guards. Not only is this in keeping with my status as an equal
opportunity employer, but it will come in handy when the hero becomes
invisible or douses my only light source."
<http://www.eviloverlord.com/>