smart shutdown at end of transaction (was: Default mode for shutdown)

On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
> It occurs to me that we may need a new mode, which disconnects sessions
> that are not in a transaction (or as soon as they are) but leaves
> in-progress transactions alone; this could be the new default.  Of
> course, this is much more difficult to implement than the current modes.

This idea appeared to have some support. I'd like to suggest that we
take this a step further. Instead of adding a fourth mode, I'd like
to suggest that we redefine "smart" to have the behavior described
above. This is based on the theory that (1) people who like smart
shutdown like it because it allows currently-running transactions to
complete without error, and will find it acceptable to have idle
transactions terminated immediately and other sessions terminated
after the command completes; and (2) people who dislike smart shutdown
(such as me) dislike it primarily because a completely idle session
that someone's forgotten to close can prevent shutdown indefinitely.
Either part of this theory could be wrong, of course, although I'm
pretty sure #2 holds for me personally at the least.

Patch is attached.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Apr 27, 2012 at 19:42, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
> <alvherre(at)commandprompt(dot)com> wrote:
>> It occurs to me that we may need a new mode, which disconnects sessions
>> that are not in a transaction (or as soon as they are) but leaves
>> in-progress transactions alone; this could be the new default.  Of
>> course, this is much more difficult to implement than the current modes.
>
> This idea appeared to have some support.  I'd like to suggest that we
> take this a step further.  Instead of adding a fourth mode, I'd like
> to suggest that we redefine "smart" to have the behavior described

+1762329!

> above.  This is based on the theory that (1) people who like smart
> shutdown like it because it allows currently-running transactions to
> complete without error, and will find it acceptable to have idle
> transactions terminated immediately and other sessions terminated

Uh, I don't think it's ok to terminate an idle transaction
immediately. An idle *session* is ok, though - maybe that's what you
mean?

Because every transaction that's *doing* multiple things will be idle
for milliseconds every now and then.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Fri, Apr 27, 2012 at 1:46 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Fri, Apr 27, 2012 at 19:42, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
>> <alvherre(at)commandprompt(dot)com> wrote:
>>> It occurs to me that we may need a new mode, which disconnects sessions
>>> that are not in a transaction (or as soon as they are) but leaves
>>> in-progress transactions alone; this could be the new default.  Of
>>> course, this is much more difficult to implement than the current modes.
>>
>> This idea appeared to have some support.  I'd like to suggest that we
>> take this a step further.  Instead of adding a fourth mode, I'd like
>> to suggest that we redefine "smart" to have the behavior described
>
> +1762329!

Thanks. :-)

>> above.  This is based on the theory that (1) people who like smart
>> shutdown like it because it allows currently-running transactions to
>> complete without error, and will find it acceptable to have idle
>> transactions terminated immediately and other sessions terminated
>
> Uh, I don't think it's ok to terminate an idle transaction
> immediately. An idle *session* is ok, though - maybe that's what you
> mean?

Yes, exactly. What the patch does is arrange things so that, when
smart shutdown is requested, we terminate each session as soon as it
is both (1) idle and (2) not in a transaction.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
> <alvherre(at)commandprompt(dot)com> wrote:
>> It occurs to me that we may need a new mode, which disconnects sessions
>> that are not in a transaction (or as soon as they are) but leaves
>> in-progress transactions alone; this could be the new default. Of
>> course, this is much more difficult to implement than the current modes.

> This idea appeared to have some support. I'd like to suggest that we
> take this a step further. Instead of adding a fourth mode, I'd like
> to suggest that we redefine "smart" to have the behavior described
> above.

No, I'm not happy with that. Smart shutdown is defined to not affect
current sessions. I'm fine with having a fourth mode that acts as you
suggest (and, probably, even with making it the default); but not with
taking away a behavior that people may well be relying on.

> This is based on the theory that (1) people who like smart
> shutdown like it because it allows currently-running transactions to
> complete without error,

I think they like it because it allows currently-running *sessions*
to complete without error. You have no real basis for asserting that
relocating that goalpost won't change the game.

regards, tom lane

On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
>> <alvherre(at)commandprompt(dot)com> wrote:
>>> It occurs to me that we may need a new mode, which disconnects sessions
>>> that are not in a transaction (or as soon as they are) but leaves
>>> in-progress transactions alone; this could be the new default.  Of
>>> course, this is much more difficult to implement than the current modes.
>
>> This idea appeared to have some support.  I'd like to suggest that we
>> take this a step further.  Instead of adding a fourth mode, I'd like
>> to suggest that we redefine "smart" to have the behavior described
>> above.
>
> No, I'm not happy with that.  Smart shutdown is defined to not affect
> current sessions.  I'm fine with having a fourth mode that acts as you
> suggest (and, probably, even with making it the default); but not with
> taking away a behavior that people may well be relying on.

Agreed, but not sure what to call the new mode: "smarter"?

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Hi,

On Friday, April 27, 2012 07:42:59 PM Robert Haas wrote:
> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
> <alvherre(at)commandprompt(dot)com> wrote:
> > It occurs to me that we may need a new mode, which disconnects sessions
> > that are not in a transaction (or as soon as they are) but leaves
> > in-progress transactions alone; this could be the new default. Of
> > course, this is much more difficult to implement than the current modes.
>
> This idea appeared to have some support. I'd like to suggest that we
> take this a step further. Instead of adding a fourth mode, I'd like
> to suggest that we redefine "smart" to have the behavior described
> above. This is based on the theory that (1) people who like smart
> shutdown like it because it allows currently-running transactions to
> complete without error, and will find it acceptable to have idle
> transactions terminated immediately and other sessions terminated
> after the command completes; and (2) people who dislike smart shutdown
> (such as me) dislike it primarily because a completely idle session
> that someone's forgotten to close can prevent shutdown indefinitely.
> Either part of this theory could be wrong, of course, although I'm
> pretty sure #2 holds for me personally at the least.
I think the current smart mode is rather useful. There is quite some stuff
that you cannot do inside a transaction - or it doesn't make sense - which
still needs to shutdown gracefully. E.g. transaction managers.

Andres

On Friday, April 27, 2012 08:38:10 PM Simon Riggs wrote:
> On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> >> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
> >>
> >> <alvherre(at)commandprompt(dot)com> wrote:
> >>> It occurs to me that we may need a new mode, which disconnects sessions
> >>> that are not in a transaction (or as soon as they are) but leaves
> >>> in-progress transactions alone; this could be the new default. Of
> >>> course, this is much more difficult to implement than the current
> >>> modes.
> >>
> >> This idea appeared to have some support. I'd like to suggest that we
> >> take this a step further. Instead of adding a fourth mode, I'd like
> >> to suggest that we redefine "smart" to have the behavior described
> >> above.
> >
> > No, I'm not happy with that. Smart shutdown is defined to not affect
> > current sessions. I'm fine with having a fourth mode that acts as you
> > suggest (and, probably, even with making it the default); but not with
> > taking away a behavior that people may well be relying on.
>
> Agreed, but not sure what to call the new mode: "smarter"?
graceful?

Andres

Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> No, I'm not happy with that. Smart shutdown is defined to not affect
>> current sessions. I'm fine with having a fourth mode that acts as you
>> suggest (and, probably, even with making it the default); but not with
>> taking away a behavior that people may well be relying on.

> Agreed, but not sure what to call the new mode: "smarter"?

I'm not necessarily opposed to commandeering the name "smart" for the
new behavior, so that what we have to find a name for is the old "smart"
behavior. How about

slow - allow existing sessions to finish (old "smart")
smart - allow existing transactions to finish (new)
fast - kill active queries
immediate - unclean shutdown

regards, tom lane

On Fri, Apr 27, 2012 at 20:48, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
>> On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> No, I'm not happy with that.  Smart shutdown is defined to not affect
>>> current sessions.  I'm fine with having a fourth mode that acts as you
>>> suggest (and, probably, even with making it the default); but not with
>>> taking away a behavior that people may well be relying on.
>
>> Agreed, but not sure what to call the new mode: "smarter"?
>
> I'm not necessarily opposed to commandeering the name "smart" for the
> new behavior, so that what we have to find a name for is the old "smart"
> behavior.  How about
>
>        slow    - allow existing sessions to finish (old "smart")

How about "wait" instead of "slow"?

>        smart   - allow existing transactions to finish (new)

and still default, right?

>        fast    - kill active queries
>        immediate - unclean shutdown

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Fri, Apr 27, 2012 at 20:48, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I'm not necessarily opposed to commandeering the name "smart" for the
>> new behavior, so that what we have to find a name for is the old "smart"
>> behavior. How about
>>
>> slow - allow existing sessions to finish (old "smart")

> How about "wait" instead of "slow"?

I kinda liked "slow" vs "fast", but if you think that's too cute ...
("wait" doesn't seem very good, though, since all these except immediate
are waiting, just for different things.)

>> smart - allow existing transactions to finish (new)

> and still default, right?

Right.

>> fast - kill active queries
>> immediate - unclean shutdown

regards, tom lane

On Fri, Apr 27, 2012 at 2:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> This idea appeared to have some support.  I'd like to suggest that we
>> take this a step further.  Instead of adding a fourth mode, I'd like
>> to suggest that we redefine "smart" to have the behavior described
>> above.
>
> No, I'm not happy with that.  Smart shutdown is defined to not affect
> current sessions.  I'm fine with having a fourth mode that acts as you
> suggest (and, probably, even with making it the default); but not with
> taking away a behavior that people may well be relying on.

I think there is no point at all in having a discussion about this
unless we can first agree that the overwhelming majority of people who
have commented on this issue on this list are unhappy with the current
default behavior. If we are not going to change the default behavior,
then there is zero point in talking about this. So I am nervous about
your use of the word "probably", because I do not want to do a bunch
of work on this just to add a fourth shutdown mode without changing
the default to something that does not suck. I would like to get some
agreement that we ARE going to change the default behavior, and then
we can argue about what exactly we're going to change it to.

>> This is based on the theory that (1) people who like smart
>> shutdown like it because it allows currently-running transactions to
>> complete without error,
>
> I think they like it because it allows currently-running *sessions*
> to complete without error.  You have no real basis for asserting that
> relocating that goalpost won't change the game.

I'm not asserting that. What I am asserting is that the vast majority
of users will consider the revised game to be more fun than the
original one.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, Apr 27, 2012 at 2:48 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> I'm not necessarily opposed to commandeering the name "smart" for the
> new behavior, so that what we have to find a name for is the old "smart"
> behavior.  How about
>
>        slow    - allow existing sessions to finish (old "smart")
>        smart   - allow existing transactions to finish (new)
>        fast    - kill active queries
>        immediate - unclean shutdown

I could live with that. Really, I'd like to have fast just be the
default. But the above compromise would still be a big improvement
over what we have now, assuming the new smart becomes the default.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 27.04.2012 21:56, Tom Lane wrote:
> Magnus Hagander<magnus(at)hagander(dot)net> writes:
>> On Fri, Apr 27, 2012 at 20:48, Tom Lane<tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> I'm not necessarily opposed to commandeering the name "smart" for the
>>> new behavior, so that what we have to find a name for is the old "smart"
>>> behavior. How about
>>>
>>> slow - allow existing sessions to finish (old "smart")
>
>> How about "wait" instead of "slow"?
>
> I kinda liked "slow" vs "fast", but if you think that's too cute ...
> ("wait" doesn't seem very good, though, since all these except immediate
> are waiting, just for different things.)

All the modes indeed wait (except for immediate), so I think it would
make sense to define the modes in terms of *what* they wait for.

wait sessions - allow existing sessions to finish (old "smart")
wait transactions - allow existing transactions to finish (new)
wait checkpoint - kill active queries
wait none - unclean shutdown

Hmm, the latter two are perhaps a bit confusing. So maybe:

wait_sessions - allow existing sessions to finish (old "smart")
wait_transactions - allow existing transactions to finish (new)
fast - kill active queries
immediate - unclean shutdown

Just thinking out loud here..

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Fri, Apr 27, 2012 at 3:00 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Fri, Apr 27, 2012 at 2:48 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I'm not necessarily opposed to commandeering the name "smart" for the
>> new behavior, so that what we have to find a name for is the old "smart"
>> behavior.  How about
>>
>>        slow    - allow existing sessions to finish (old "smart")
>>        smart   - allow existing transactions to finish (new)
>>        fast    - kill active queries
>>        immediate - unclean shutdown
>
> I could live with that.  Really, I'd like to have fast just be the
> default.  But the above compromise would still be a big improvement
> over what we have now, assuming the new smart becomes the default.

So right now, we have a mapping from signals to shutdown types that
looks like this:

[Current] SIGTERM -> smart, SIGINT -> fast, SIGQUIT -> immediate

It seems we need another signal for the new mode, and the obvious
candidate is SIGUSR2. But what shall the mapping look like?

[Choice #1] SIGUSR2 -> slow, SIGTERM -> smart, SIGINT -> fast, SIGQUIT
-> immediate
[Choice #2] SIGTERM -> slow, SIGUSR2 -> smart, SIGINT -> fast, SIGQUIT
-> immediate

In other words, should we retain the existing behavior for SIGTERM and
make SIGUSR2 have the new behavior (choice #2)? Or shall we preserve
the invariant that SIGTERM invokes the default shutdown mode, and move
the current default behavior off into SIGUSR2 land (choice #1)?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:

> Just thinking out loud here..

In the spirit of kicking around ideas...

For those writing service scripts where you want a time limit on how
long a stop can take, so that the service script doesn't prevent OS
shutdown within a bounded time, it would also be nice to add an
escalation time limit; so if there isn't a shutdown withing k
seconds at one level it goes to the next. If the building is on
fire and you need to power down all equipment before the fire
department cuts power and starts spraying water (a situation we had
at a courthouse a year or two ago), you really don't want the OS
waiting for anything for more than a limited number of seconds
before escalating to immediate. We do that in our sh scripts now,
by using kill and sleep instead of trusting pg_ctl, but it seems
like it would be better to have pg_ctl know how to do that.

maybe?:

--escalate-after=seconds

-Kevin

On fre, 2012-04-27 at 20:39 +0200, Andres Freund wrote:
> I think the current smart mode is rather useful. There is quite some
> stuff that you cannot do inside a transaction - or it doesn't make
> sense - which still needs to shutdown gracefully. E.g. transaction
> managers.

Could you elaborate on that? What would happen to the transaction
manager if you terminate any idle, not-in-a-transaction database backend
sessions it has established?

On Friday, April 27, 2012 10:17:59 PM Peter Eisentraut wrote:
> On fre, 2012-04-27 at 20:39 +0200, Andres Freund wrote:
> > I think the current smart mode is rather useful. There is quite some
> > stuff that you cannot do inside a transaction - or it doesn't make
> > sense - which still needs to shutdown gracefully. E.g. transaction
> > managers.
> Could you elaborate on that? What would happen to the transaction
> manager if you terminate any idle, not-in-a-transaction database backend
> sessions it has established?
In the few cases where I investigated it TMs don't use transactions themselves
(which I think is correct, they don't need them), so terminating any idle
session - which the TM would appear as, as its not using txns - would leave
prepared transactions in a limbo state till the database is up again, instead
of waiting till all prepared transactions are either aborted or committed. It
may also choose to coordinate to abort all transactions, but all that is hard
if the database shuts you out.
I actually also have co-maintained other software where some processes have an
idle connection open on which some shutdown stuff will happen. Obviously all
those will need to handle the case where the connection was aborted, but that
may result in suboptimal behaviour. Requiring such processes to keep open a
transaction doesn't seem to be a good design choice in the pg world.

Andres

On Fri, Apr 27, 2012 at 1:57 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> I think there is no point at all in having a discussion about this
> unless we can first agree that the overwhelming majority of people who
> have commented on this issue on this list are unhappy with the current
> default behavior.

count me in. the current behavior sucks.

merlin

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> It seems we need another signal for the new mode, and the obvious
> candidate is SIGUSR2. But what shall the mapping look like?

> [Choice #1] SIGUSR2 -> slow, SIGTERM -> smart, SIGINT -> fast, SIGQUIT
> -> immediate
> [Choice #2] SIGTERM -> slow, SIGUSR2 -> smart, SIGINT -> fast, SIGQUIT
> -> immediate

SIGTERM needs to correspond to a fairly aggressive shutdown mode,
since (at least on some systems) init will send that during the system
shutdown sequence, shortly before escalating to SIGKILL. So I think
choice #2 is not sensible at all.

If we were willing to consider wholesale breakage of any scripts that
send these signals directly, I'd almost consider that it should be
SIGUSR2, SIGINT, SIGTERM, SIGQUIT. But that might be more churn than
we want. Keeping SIGTERM attached to the default/"smart" shutdown mode
seems like a reasonable compromise.

regards, tom lane

On fre, 2012-04-27 at 22:30 +0200, Andres Freund wrote:
> In the few cases where I investigated it TMs don't use transactions
> themselves (which I think is correct, they don't need them), so
> terminating any idle session - which the TM would appear as, as its
> not using txns - would leave prepared transactions in a limbo state
> till the database is up again, instead of waiting till all prepared
> transactions are either aborted or committed. It may also choose to
> coordinate to abort all transactions, but all that is hard if the
> database shuts you out.

This would lead to another shutdown mode, one that terminates idle
sessions unless they have prepared transactions. That could be useful.

On fre, 2012-04-27 at 18:09 -0400, Tom Lane wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> > It seems we need another signal for the new mode, and the obvious
> > candidate is SIGUSR2. But what shall the mapping look like?
>
> > [Choice #1] SIGUSR2 -> slow, SIGTERM -> smart, SIGINT -> fast, SIGQUIT
> > -> immediate
> > [Choice #2] SIGTERM -> slow, SIGUSR2 -> smart, SIGINT -> fast, SIGQUIT
> > -> immediate
>
> SIGTERM needs to correspond to a fairly aggressive shutdown mode,
> since (at least on some systems) init will send that during the system
> shutdown sequence, shortly before escalating to SIGKILL.

That only happens if the postgresql init script itself didn't do a good
job. We already have this setup currently, and it doesn't seem to cause
a great deal of problems.

> If we were willing to consider wholesale breakage of any scripts that
> send these signals directly, I'd almost consider that it should be
> SIGUSR2, SIGINT, SIGTERM, SIGQUIT. But that might be more churn than
> we want. Keeping SIGTERM attached to the default/"smart" shutdown mode
> seems like a reasonable compromise.

I don't think we should change the traditional "severity" order of
signals.

On Fri, Apr 27, 2012 at 7:57 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> I think there is no point at all in having a discussion about this
> unless we can first agree that the overwhelming majority of people who
> have commented on this issue on this list are unhappy with the current
> default behavior.  If we are not going to change the default behavior,
> then there is zero point in talking about this.

That doesn't follow.

You are right to bring up this issue. Many people do think current
"smart" mode is annoying, though we must accept that some people like
it *and* that changing the default behaviour in one release is a bad
thing.

I don't think anyone has spoken against introducing a new mode. Having
it is a good thing, whether or not it is default.

So lets implement the new shutdown mode and work out a transition path
to a new default. Changing rapidly screws up the people we love the
most.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

On Fri, Apr 27, 2012 at 8:36 PM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:

> All the modes indeed wait (except for immediate), so I think it would make
> sense to define the modes in terms of *what* they wait for.
>
>        wait sessions   - allow existing sessions to finish (old "smart")
>        wait transactions       - allow existing transactions to finish (new)
>        wait checkpoint - kill active queries
>        wait none - unclean shutdown
>
> Hmm, the latter two are perhaps a bit confusing. So maybe:
>
>        wait_sessions   - allow existing sessions to finish (old "smart")
>        wait_transactions       - allow existing transactions to finish (new)
>
>        fast    - kill active queries
>        immediate - unclean shutdown
>
> Just thinking out loud here..

+1

Wonderfully clear, little need to check the docs to see what the terms
actually mean.

New names for both allow us to deprecate use of "smart", since it was
a silly term anyway. We keep smart for one more
release==wait_sessions, then throw an error in later releases.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On fre, 2012-04-27 at 22:30 +0200, Andres Freund wrote:
>> In the few cases where I investigated it TMs don't use transactions
>> themselves (which I think is correct, they don't need them), so
>> terminating any idle session - which the TM would appear as, as its
>> not using txns - would leave prepared transactions in a limbo state
>> till the database is up again, instead of waiting till all prepared
>> transactions are either aborted or committed. It may also choose to
>> coordinate to abort all transactions, but all that is hard if the
>> database shuts you out.

> This would lead to another shutdown mode, one that terminates idle
> sessions unless they have prepared transactions. That could be useful.

Huh? Prepared transactions aren't associated with sessions. At least
not in a context using a TM --- the TM will be doing commits or
rollbacks from a session different from the ones that ran the prepared
transactions.

regards, tom lane

On Sat, Apr 28, 2012 at 7:04 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On Fri, Apr 27, 2012 at 7:57 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> I think there is no point at all in having a discussion about this
>> unless we can first agree that the overwhelming majority of people who
>> have commented on this issue on this list are unhappy with the current
>> default behavior.  If we are not going to change the default behavior,
>> then there is zero point in talking about this.
>
> That doesn't follow.
>
> You are right to bring up this issue. Many people do think current
> "smart" mode is annoying, though we must accept that some people like
> it *and* that changing the default behaviour in one release is a bad
> thing.
>
> I don't think anyone has spoken against introducing a new mode. Having
> it is a good thing, whether or not it is default.
>
> So lets implement the new shutdown mode and work out a transition path
> to a new default. Changing rapidly screws up the people we love the
> most.

In some cases, there are ways to phase in a change over a series of
releases, but I don't see how that would be possible here. If we
intend ever to change the default mode, then we have to do it
sometime, and that release is going to have a backward-incompatibility
no matter which one it is. Personally, as backward incompatibilities
go, I think this one is pretty minor. Most people are probably
already using scripts that specify fast mode, and those scripts won't
change. But even for people who actually are using smart mode, most
people do not shut down the database all that often, and it's rather
pessimistic to suppose that the proposed new mode will break anything
for them. But even if it does, we can't make improvements to the
system without sometimes changing things in a backward-incompatible
way, and if we get into the mind-set that no amount of
backward-incompatibility is ever acceptable, we're going to seriously
limit our opportunities to revisit poor design decisions.

I think there's a funny kind of thing that happens when we discuss a
behavior that is sub-optimal: we start to look for ways to justify
leaving it the way it is, because surely it couldn't be a terrible
idea if it's been like that forever. I think there's some of that
going on on the thread about stripping trailing null columns, too: if
we've got a benchmark result showing that the patch saves CPU time on
a 5-column table (!), then all the pontificating about 700-column
tables being rare is irrelevant. Similarly here: it's true that
someone might have to revisit their init scripts, but should they fail
to do so, the consequences are really not that dire.

On the other hand, in PostgreSQL 8.4, we changed TRUNCATE to wipe out
the entire inheritance hierarchy instead of only the named table
(unless the new ONLY keyword was added). This obviously has the
potential to be completely disastrous for someone with a very
particular usage pattern, but there was little discussion and everyone
basically said "yeah, we should go ahead and change that, despite the
small risk that someone will accidentally blow away a lot more data
than they intended". Maybe there are more people using smart shutdown
than there are people truncating only the root of an inheritance
hierarchy, but nothing we're proposing to do here is going to
permanently erase anyone's data, either.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On lör, 2012-04-28 at 11:12 -0400, Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > On fre, 2012-04-27 at 22:30 +0200, Andres Freund wrote:
> >> In the few cases where I investigated it TMs don't use transactions
> >> themselves (which I think is correct, they don't need them), so
> >> terminating any idle session - which the TM would appear as, as its
> >> not using txns - would leave prepared transactions in a limbo state
> >> till the database is up again, instead of waiting till all prepared
> >> transactions are either aborted or committed. It may also choose to
> >> coordinate to abort all transactions, but all that is hard if the
> >> database shuts you out.
>
> > This would lead to another shutdown mode, one that terminates idle
> > sessions unless they have prepared transactions. That could be useful.
>
> Huh? Prepared transactions aren't associated with sessions. At least
> not in a context using a TM --- the TM will be doing commits or
> rollbacks from a session different from the ones that ran the prepared
> transactions.

From what Andres wrote I gather that the TM would be using the same
session for preparing and committing.

In any case, if either the existing session of the TM is cut or it
cannot create a new connection, it will, after some time, have to give
up roll back the prepared transactions on the other servers. So some
kind of setting to not shut down if there are prepared transactions
pending could be useful. But this could probably be a separate GUC
setting or two instead of a shutdown mode (or two) of its own.

On fre, 2012-04-27 at 14:57 -0400, Robert Haas wrote:
> I think there is no point at all in having a discussion about this
> unless we can first agree that the overwhelming majority of people who
> have commented on this issue on this list are unhappy with the current
> default behavior. If we are not going to change the default behavior,
> then there is zero point in talking about this.

Have you reviewed the previous discussions where changing the default
behavior was discussed and rejected? I don't like the current default
any more than you do, but without any new arguments, there is, as you
say, zero point in talking about this.

On Sun, Apr 29, 2012 at 12:41 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Sat, Apr 28, 2012 at 7:04 AM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

>> So lets implement the new shutdown mode and work out a transition path
>> to a new default. Changing rapidly screws up the people we love the
>> most.
>
> In some cases, there are ways to phase in a change over a series of
> releases, but I don't see how that would be possible here.  If we
> intend ever to change the default mode, then we have to do it
> sometime, and that release is going to have a backward-incompatibility
> no matter which one it is.  Personally, as backward incompatibilities
> go, I think this one is pretty minor.  Most people are probably
> already using scripts that specify fast mode, and those scripts won't
> change.  But even for people who actually are using smart mode, most
> people do not shut down the database all that often, and it's rather
> pessimistic to suppose that the proposed new mode will break anything
> for them.  But even if it does, we can't make improvements to the
> system without sometimes changing things in a backward-incompatible
> way, and if we get into the mind-set that no amount of
> backward-incompatibility is ever acceptable, we're going to seriously
> limit our opportunities to revisit poor design decisions.
>
> I think there's a funny kind of thing that happens when we discuss a
> behavior that is sub-optimal: we start to look for ways to justify
> leaving it the way it is, because surely it couldn't be a terrible
> idea if it's been like that forever.  I think there's some of that
> going on on the thread about stripping trailing null columns, too: if
> we've got a benchmark result showing that the patch saves CPU time on
> a 5-column table (!), then all the pontificating about 700-column
> tables being rare is irrelevant.  Similarly here: it's true that
> someone might have to revisit their init scripts, but should they fail
> to do so, the consequences are really not that dire.
>
> On the other hand, in PostgreSQL 8.4, we changed TRUNCATE to wipe out
> the entire inheritance hierarchy instead of only the named table
> (unless the new ONLY keyword was added).  This obviously has the
> potential to be completely disastrous for someone with a very
> particular usage pattern, but there was little discussion and everyone
> basically said "yeah, we should go ahead and change that, despite the
> small risk that someone will accidentally blow away a lot more data
> than they intended".  Maybe there are more people using smart shutdown
> than there are people truncating only the root of an inheritance
> hierarchy, but nothing we're proposing to do here is going to
> permanently erase anyone's data, either.

I don't think you can use the TRUNCATE case as an example. For me,
that was a prime case of insufficient discussion around the principle
of backwards compatibility. It wasn't clear to me that was happening
and had I known, I would have objected. IIRC the first I knew of it
was when the release notes came out months after things were settled.

We go to great lengths to note initdb inducing behaviour during beta,
but very little towards behaviour changes that require downstream
software changes.

Maybe we don't need to do this over multiple releases, but we do need
to give warning of possible incompatibilities. It would be good to see
a specific post on hackers called "Planned Incompatibilities in 9.2",
or collect such things on the open items wiki, so that people
listening can see what might happen and get a chance to object. Or if
changes do go ahead, at least we give them a few months warning to
change the downstream software. Otherwise all that happens is our new
release comes out and fewer people use it because it takes ages to
actually realign the software stack enough for our software to be
used.

The better we succeed at persuading the world to use Postgres the more
important backwards compatibility becomes. When fewer people used
Postgres it was easy to charge forwards aggressively, but as we begin
to lead we must be more careful.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> In any case, if either the existing session of the TM is cut or it
> cannot create a new connection, it will, after some time, have to give
> up roll back the prepared transactions on the other servers. So some
> kind of setting to not shut down if there are prepared transactions
> pending could be useful. But this could probably be a separate GUC
> setting or two instead of a shutdown mode (or two) of its own.

This argument still seems pretty bogus. The *entire* point of a TM
is to cope with crashes of individual databases under its management.
The proposed setting seems to amount to a "please don't crash" GUC,
which is silly on its face, and does not actually make the TM's life
any easier anyway.

regards, tom lane

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On fre, 2012-04-27 at 14:57 -0400, Robert Haas wrote:
>> I think there is no point at all in having a discussion about this
>> unless we can first agree that the overwhelming majority of people who
>> have commented on this issue on this list are unhappy with the current
>> default behavior. If we are not going to change the default behavior,
>> then there is zero point in talking about this.

> Have you reviewed the previous discussions where changing the default
> behavior was discussed and rejected? I don't like the current default
> any more than you do, but without any new arguments, there is, as you
> say, zero point in talking about this.

Perhaps I've forgotten something, but I only recall debates about
switching the default to a different one of the existing shutdown modes.
The new material here is the proposal for a new mode.

regards, tom lane

On Sun, Apr 29, 2012 at 4:06 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>> In any case, if either the existing session of the TM is cut or it
>> cannot create a new connection, it will, after some time, have to give
>> up roll back the prepared transactions on the other servers.  So some
>> kind of setting to not shut down if there are prepared transactions
>> pending could be useful.  But this could probably be a separate GUC
>> setting or two instead of a shutdown mode (or two) of its own.
>
> This argument still seems pretty bogus.  The *entire* point of a TM
> is to cope with crashes of individual databases under its management.
> The proposed setting seems to amount to a "please don't crash" GUC,
> which is silly on its face, and does not actually make the TM's life
> any easier anyway.

You are right that the TM can cope with aborted transactions, but that
doesn't mean we should force it to have to do that. If we can wait for
commit then we should do so.

I think we only need one new mode, "shutdown when transactions are
finished" should only shutdown when all types of transaction are
complete. For people that don't use prepared transactions the
difference is irrelevant. For people that do use prepared
transactions, I can't imagine they would want a new setting that ends
with aborted transactions, since that isn't any different to a fast
shutdown.

If that hangs waiting for TM that has gone away, then you can issue
shutdown fast.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> I think we only need one new mode, "shutdown when transactions are
> finished" should only shutdown when all types of transaction are
> complete. For people that don't use prepared transactions the
> difference is irrelevant. For people that do use prepared
> transactions, I can't imagine they would want a new setting that ends
> with aborted transactions, since that isn't any different to a fast
> shutdown.

That sounds reasonable at first blush. Implementing it might be
trickier than you think though, since (despite Peter's opinion) the
prepared xacts are not associated with any particular session, and the
postmaster itself doesn't know they are there. What's more, if
individual sessions are told to commit hara-kiri as soon as they are not
in a transaction, there soon won't be any surviving session in which the
TM could issue a COMMIT PREPARED.

I think the only way this could be made to fly would be if the TM could
set a session state that indicates "I'm a TM session, don't kill me
until all prepared transactions are gone". Which might be problematic
from a security standpoint, if random users could use it to proof
themselves against getting kicked out. We could make it SUSET but then
TMs would have to run as superuser, which seems a bit less than
desirable.

On the whole it is not apparent to me that we really need a mode in
which shutdown waits for prepared transactions to flush out; and I would
definitely not be in favor of it being the default. I think that that
would make prepared transactions an even bigger foot-gun than they are
now. Just think: you say "pg_ctl stop", and the server promptly kicks
off all your users and won't let any more in, but doesn't actually shut
down. You may not know why, and even if you do, you can't connect to do
something about it. Eventually you give up and issue shutdown fast,
cursing whoever designed that misbegotten behavior.

regards, tom lane

On Sun, Apr 29, 2012 at 5:41 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
>> I think we only need one new mode, "shutdown when transactions are
>> finished" should only shutdown when all types of transaction are
>> complete. For people that don't use prepared transactions the
>> difference is irrelevant. For people that do use prepared
>> transactions, I can't imagine they would want a new setting that ends
>> with aborted transactions, since that isn't any different to a fast
>> shutdown.
>
> That sounds reasonable at first blush.  Implementing it might be
> trickier than you think though, since (despite Peter's opinion) the
> prepared xacts are not associated with any particular session, and the
> postmaster itself doesn't know they are there.  What's more, if
> individual sessions are told to commit hara-kiri as soon as they are not
> in a transaction, there soon won't be any surviving session in which the
> TM could issue a COMMIT PREPARED.
>
> I think the only way this could be made to fly would be if the TM could
> set a session state that indicates "I'm a TM session, don't kill me
> until all prepared transactions are gone".  Which might be problematic
> from a security standpoint, if random users could use it to proof
> themselves against getting kicked out.  We could make it SUSET but then
> TMs would have to run as superuser, which seems a bit less than
> desirable.

I think an explicit state is overkill and has other problems as you say.

> On the whole it is not apparent to me that we really need a mode in
> which shutdown waits for prepared transactions to flush out; and I would
> definitely not be in favor of it being the default.  I think that that
> would make prepared transactions an even bigger foot-gun than they are
> now.  Just think: you say "pg_ctl stop", and the server promptly kicks
> off all your users and won't let any more in, but doesn't actually shut
> down.  You may not know why, and even if you do, you can't connect to do
> something about it.  Eventually you give up and issue shutdown fast,
> cursing whoever designed that misbegotten behavior.

Waiting too long is clearly a foot fun, as you say.

But if you just issued PREPARE on a session, its more than likely that
this will be followed almost immediately by a COMMIT. Simply waiting
is a good indication, and some reasonable time like 10 seconds is fine
in determining whether that COMMIT will arrive, or not.

This only matters on a shutdown. If its a restart, we can shutdown
after a PREPARE because as soon as we are back up again the TM can
issue the COMMIT.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

On sön, 2012-04-29 at 10:19 +0100, Simon Riggs wrote:
> Maybe we don't need to do this over multiple releases, but we do need
> to give warning of possible incompatibilities. It would be good to see
> a specific post on hackers called "Planned Incompatibilities in 9.2",
> or collect such things on the open items wiki, so that people
> listening can see what might happen and get a chance to object. Or if
> changes do go ahead, at least we give them a few months warning to
> change the downstream software. Otherwise all that happens is our new
> release comes out and fewer people use it because it takes ages to
> actually realign the software stack enough for our software to be
> used.

Well, either there are possible incompatibilities, in which case users
will be slow to adopt new releases, as is currently the case, or there
strictly won't be any (unless hidden behind config settings or similar),
but then introducing new features or bug fixes can take many years. So
far we've erred on the side of "progress".

On Sun, Apr 29, 2012 at 1:48 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On sön, 2012-04-29 at 10:19 +0100, Simon Riggs wrote:
>> Maybe we don't need to do this over multiple releases, but we do need
>> to give warning of possible incompatibilities. It would be good to see
>> a specific post on hackers called "Planned Incompatibilities in 9.2",
>> or collect such things on the open items wiki, so that people
>> listening can see what might happen and get a chance to object. Or if
>> changes do go ahead, at least we give them a few months warning to
>> change the downstream software. Otherwise all that happens is our new
>> release comes out and fewer people use it because it takes ages to
>> actually realign the software stack enough for our software to be
>> used.
>
> Well, either there are possible incompatibilities, in which case users
> will be slow to adopt new releases, as is currently the case, or there
> strictly won't be any (unless hidden behind config settings or similar),
> but then introducing new features or bug fixes can take many years.  So
> far we've erred on the side of "progress".

"Erred on the side of progress" might even be a little strong, because
I think for the most part we have been extremely judicious about
backward incompatibilities in the last few releases (which is a good
thing). Obviously, 8.3 was a flag day of the first magnitude, and one
I hope we won't repeat any time soon, but if you look through the
release notes for, say, 9.1, just about every "incompatibility" listed
there amounts to fixing something that was either demonstrably broken
or widely hated in prior releases. Turning on
standard_conforming_strings by default was a big deal, but we've been
phasing that change in for five years or so, so I think we really did
about as much to ease that transition as is humanly possible.
Moreover, you can always turn the GUC off again, if the new behaviour
is a problem.

The only way we're going to have fewer incompatibilities than we do
now is to preserve existing behavior even when it's broken,
widely-hated, and/or not standards-conformant. IMHO, that would be
taking a sound principle to an illogical extreme.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> "Erred on the side of progress" might even be a little strong, because
> I think for the most part we have been extremely judicious about
> backward incompatibilities in the last few releases (which is a good
> thing). Obviously, 8.3 was a flag day of the first magnitude, and one
> I hope we won't repeat any time soon, but if you look through the
> release notes for, say, 9.1, just about every "incompatibility" listed
> there amounts to fixing something that was either demonstrably broken
> or widely hated in prior releases.

Well, if you're ragging on the implicit coercions changes, let me point
out that those were also fixing something that was demonstrably broken.
So I'm afraid it's a tad pollyanna-ish to claim that there is never
going to be push-back on changes we choose to make for one or another
of these reasons.

Having said that, though, I agree that we have to be willing to make
incompatible changes from time to time, and I think our standards for
when to do that are plenty high enough already. I'm not in favor of
raising that bar still more. The reason we support back branches as
long as we do is precisely to give people the option to not deal with
incompatible changes until they are ready to. I don't think we need
to do even more, and I don't want to add still more overhead to the
development process to do it.

regards, tom lane

On Sun, Apr 29, 2012 at 5:27 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> "Erred on the side of progress" might even be a little strong, because
>> I think for the most part we have been extremely judicious about
>> backward incompatibilities in the last few releases (which is a good
>> thing).  Obviously, 8.3 was a flag day of the first magnitude, and one
>> I hope we won't repeat any time soon, but if you look through the
>> release notes for, say, 9.1, just about every "incompatibility" listed
>> there amounts to fixing something that was either demonstrably broken
>> or widely hated in prior releases.
>
> Well, if you're ragging on the implicit coercions changes, let me point
> out that those were also fixing something that was demonstrably broken.

True, but it was painful for a lot of people, and I continue to think
that we broke too many reasonable cases.

> So I'm afraid it's a tad pollyanna-ish to claim that there is never
> going to be push-back on changes we choose to make for one or another
> of these reasons.

Agreed, I expect some push-back. I'm just pointing out that we reject
a very significant number of changes on backward-compatibility
grounds. We don't reject too many entire patches on these grounds,
but many are the patch authors who have been asked to change X,Y, or Z
to avoid breaking backward compatibility, or who have had things
ripped out by the committer for such reasons. Of course this is
sometimes an occasion for complaint, and then the backward
compatibility changes that do get through are also an occasion for
complaint, so there's no perfect world, but we do try pretty hard, I
think.

> Having said that, though, I agree that we have to be willing to make
> incompatible changes from time to time, and I think our standards for
> when to do that are plenty high enough already.  I'm not in favor of
> raising that bar still more.  The reason we support back branches as
> long as we do is precisely to give people the option to not deal with
> incompatible changes until they are ready to.  I don't think we need
> to do even more, and I don't want to add still more overhead to the
> development process to do it.

+1, and well put.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Tom Lane wrote:
>> On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> No, I'm not happy with that. Smart shutdown is defined to not
affect
>>> current sessions. I'm fine with having a fourth mode that acts as
you
>>> suggest (and, probably, even with making it the default); but not
with
>>> taking away a behavior that people may well be relying on.

>> Agreed, but not sure what to call the new mode: "smarter"?

> I'm not necessarily opposed to commandeering the name "smart" for the
> new behavior, so that what we have to find a name for is the old
"smart"
> behavior. How about
>
> slow - allow existing sessions to finish (old "smart")
> smart - allow existing transactions to finish (new)
> fast - kill active queries
> immediate - unclean shutdown

But if the meaning of "smart" changes, then people who use
"pg_ctl stop -m smart" and expect that active sessions will not be
affected will get a surprise.

Wouldn't it be better to pick a different name for the new fourth
mode? It could still be the default mode, but I think that people
who explicitly specify a certain mode are more likely to care about
the exact behaviour.

I second Heikki's suggestions for mode names.

And +1 from me on changing the default behaviour.

Yours,
Laurenz Albe

Just for the ones interested in a view on another turf:

In Oracle "shutdown immediate" is the fastest _clean_ shutdown and "shutdown abort" is equal to "shutdown immediate" in PG.
The other modes are called "shutdown normal" and "shutdown transactional".

Wolfgang

________________________________
Von: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
An: Simon Riggs <simon(at)2ndQuadrant(dot)com>
CC: Robert Haas <robertmhaas(at)gmail(dot)com>; Alvaro Herrera <alvherre(at)commandprompt(dot)com>; Magnus Hagander <magnus(at)hagander(dot)net>; PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Gesendet: 20:48 Freitag, 27.April 2012
Betreff: Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

Simon Riggs <simon(at)2ndQuadrant(dot)com> writes:
> On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> No, I'm not happy with that.  Smart shutdown is defined to not affect
>> current sessions.  I'm fine with having a fourth mode that acts as you
>> suggest (and, probably, even with making it the default); but not with
>> taking away a behavior that people may well be relying on.

> Agreed, but not sure what to call the new mode: "smarter"?

I'm not necessarily opposed to commandeering the name "smart" for the
new behavior, so that what we have to find a name for is the old "smart"
behavior.  How about

    slow    - allow existing sessions to finish (old "smart")
    smart    - allow existing transactions to finish (new)
    fast    - kill active queries
    immediate - unclean shutdown

            regards, tom lane

On Mon, Apr 30, 2012 at 9:55 AM, Wolfgang Wilhelm
<wolfgang20121964(at)yahoo(dot)de> wrote:
> Just for the ones interested in a view on another turf:
>
> In Oracle "shutdown immediate" is the fastest _clean_ shutdown and "shutdown
> abort" is equal to "shutdown immediate" in PG.
> The other modes are called "shutdown normal" and "shutdown transactional".

Though the behaviour users see is quite different. In Oracle the
fastest clean shutdown still requires rolling back transactions which
can take a long time. In Postgres rolling back transactions is
instantaneous so a shutdown immediate will appear to behave like a
shutdown abort in Oracle in that it will always run fast even if the
effect on the database is different.

--
greg

On Sun, Apr 29, 2012 at 10:19:38AM +0100, Simon Riggs wrote:
> Maybe we don't need to do this over multiple releases, but we do need
> to give warning of possible incompatibilities. It would be good to see
> a specific post on hackers called "Planned Incompatibilities in 9.2",
> or collect such things on the open items wiki, so that people
> listening can see what might happen and get a chance to object. Or if
> changes do go ahead, at least we give them a few months warning to
> change the downstream software. Otherwise all that happens is our new
> release comes out and fewer people use it because it takes ages to
> actually realign the software stack enough for our software to be
> used.

The release notes would certainly feature this as an incompatibility,
and would be present even before beta started. Unless they skip reading
the release notes, it would be hard for them to miss this change. I
also blog when major release notes are available for viewing.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Sat, Apr 28, 2012 at 4:00 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Fri, Apr 27, 2012 at 2:48 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I'm not necessarily opposed to commandeering the name "smart" for the
>> new behavior, so that what we have to find a name for is the old "smart"
>> behavior.  How about
>>
>>        slow    - allow existing sessions to finish (old "smart")
>>        smart   - allow existing transactions to finish (new)
>>        fast    - kill active queries
>>        immediate - unclean shutdown
>
> I could live with that.  Really, I'd like to have fast just be the
> default.  But the above compromise would still be a big improvement
> over what we have now, assuming the new smart becomes the default.

Should this new shutdown mode wait for online backup like old "smart" does?

Regards,

--
Fujii Masao

Fujii Masao wrote:
>>> I'm not necessarily opposed to commandeering the name "smart" for the
>>> new behavior, so that what we have to find a name for is the old "smart"
>>> behavior.  How about
>>>
>>>        slow    - allow existing sessions to finish (old "smart")
>>>        smart   - allow existing transactions to finish (new)
>>>        fast    - kill active queries
>>>        immediate - unclean shutdown
>>
>> I could live with that.  Really, I'd like to have fast just be the
>> default.  But the above compromise would still be a big improvement
>> over what we have now, assuming the new smart becomes the default.
>
> Should this new shutdown mode wait for online backup like old "smart" does?

I think it shouldn't; I like to think of it as some kind of "quite fast"
shutdown (provided there are no long-running transactions).

And I still think that we should choose a name different from "smart"
to indicate that something has changed, even if it is the new default.

Yours,
Laurenz Albe

On Sat, May 5, 2012 at 12:41 PM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> On Sat, Apr 28, 2012 at 4:00 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Fri, Apr 27, 2012 at 2:48 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> I'm not necessarily opposed to commandeering the name "smart" for the
>>> new behavior, so that what we have to find a name for is the old "smart"
>>> behavior.  How about
>>>
>>>        slow    - allow existing sessions to finish (old "smart")
>>>        smart   - allow existing transactions to finish (new)
>>>        fast    - kill active queries
>>>        immediate - unclean shutdown
>>
>> I could live with that.  Really, I'd like to have fast just be the
>> default.  But the above compromise would still be a big improvement
>> over what we have now, assuming the new smart becomes the default.
>
> Should this new shutdown mode wait for online backup like old "smart" does?

I think it had better not, because what happens when all the
connections are gone, no new ones can be made, and yet online backup
mode is still active?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, May 8, 2012 at 12:59 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Sat, May 5, 2012 at 12:41 PM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>> On Sat, Apr 28, 2012 at 4:00 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>> On Fri, Apr 27, 2012 at 2:48 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>>> I'm not necessarily opposed to commandeering the name "smart" for the
>>>> new behavior, so that what we have to find a name for is the old "smart"
>>>> behavior.  How about
>>>>
>>>>        slow    - allow existing sessions to finish (old "smart")
>>>>        smart   - allow existing transactions to finish (new)
>>>>        fast    - kill active queries
>>>>        immediate - unclean shutdown
>>>
>>> I could live with that.  Really, I'd like to have fast just be the
>>> default.  But the above compromise would still be a big improvement
>>> over what we have now, assuming the new smart becomes the default.
>>
>> Should this new shutdown mode wait for online backup like old "smart" does?
>
> I think it had better not, because what happens when all the
> connections are gone, no new ones can be made, and yet online backup
> mode is still active?

Yep, I agree that new mode should not. This change of the default shutdown
behavior might surprise some users, so it's better to document also this in
release note.

Regards,

--
Fujii Masao