Re: Default mode for shutdown

I'm sure this has been up before, but hey, let's take it another round.

Why don't we change the default shutdown mode for pg_ctl from "smart"
to "fast"? I've never come across a single usecase where "smart" is
what people *want*... Not sure if others have?

Yes, I realize it's somewhat of a backwards compatibility thing - but
it will at least not change things for most packages, since I believe
all those use "fast" anyway.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> I'm sure this has been up before, but hey, let's take it another round.
> Why don't we change the default shutdown mode for pg_ctl from "smart"
> to "fast"? I've never come across a single usecase where "smart" is
> what people *want*...

Really? Personally I'm quite happy with that default.

regards, tom lane

On Wed, Dec 15, 2010 at 9:39 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> I'm sure this has been up before, but hey, let's take it another round.
>> Why don't we change the default shutdown mode for pg_ctl from "smart"
>> to "fast"? I've never come across a single usecase where "smart" is
>> what people *want*...

+1. I think we should either have a timeout for "smart" shutdown mode
such that it turns into a fast shutdown after a configurable number of
seconds that defaults to, say, 30; or we should just make the default
fast shutdown as proposed.

> Really?  Personally I'm quite happy with that default.

Why? It seems to me that just leads to, oh, gee, the database isn't
shutting down, where's the window where I failed to exit a session?
And it's even worse in production, where whatever you're using for
connection pooling ensures that shutdown will take, if not forever, at
least a very, very long time.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Dec 15, 2010 at 09:39:12AM -0500, Tom Lane wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
> > I'm sure this has been up before, but hey, let's take it another round.
> > Why don't we change the default shutdown mode for pg_ctl from "smart"
> > to "fast"? I've never come across a single usecase where "smart" is
> > what people *want*...
>
> Really? Personally I'm quite happy with that default.
>
> regards, tom lane
>
+1

I think the default is perfect. Even if the usecase that is wanted
is "fast", it should be requested each time to verify that a more
destructive shutdown is wanted. If it is really an issue, a script
or shell alias can be defined to perform the more aggressive
shutdown processes.

Regards,
Ken

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Wed, Dec 15, 2010 at 9:39 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Really? Personally I'm quite happy with that default.

> Why? It seems to me that just leads to, oh, gee, the database isn't
> shutting down, where's the window where I failed to exit a session?

Yeah, and more to the point, do I want to finish whatever I was doing in
that window? Fast-by-default is a nice hammer to swing, but one day
you'll pound your finger.

regards, tom lane

On Wed, Dec 15, 2010 at 9:47 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Wed, Dec 15, 2010 at 9:39 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Really?  Personally I'm quite happy with that default.
>
>> Why?  It seems to me that just leads to, oh, gee, the database isn't
>> shutting down, where's the window where I failed to exit a session?
>
> Yeah, and more to the point, do I want to finish whatever I was doing in
> that window?  Fast-by-default is a nice hammer to swing, but one day
> you'll pound your finger.

I guess. I've pounded my finger enough time with the current default
that I'd be willing to try a different size hammer. The scenario you
describe has yet to occur in 10+ years of using the product, but
obviously not everyone's experience will match on this point.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Wed, Dec 15, 2010 at 9:47 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Yeah, and more to the point, do I want to finish whatever I was doing in
>> that window? Fast-by-default is a nice hammer to swing, but one day
>> you'll pound your finger.

> I guess. I've pounded my finger enough time with the current default
> that I'd be willing to try a different size hammer. The scenario you
> describe has yet to occur in 10+ years of using the product, but
> obviously not everyone's experience will match on this point.

I think the ultimate basis for the way it's set up now is the mantra of
"be safe by default"; which I believe I've heard you repeating in other
contexts. Between that principle and the backwards-compatibility
hazards, I really don't think there's adequate justification for
changing this.

regards, tom lane

On Wed, Dec 15, 2010 at 9:57 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Wed, Dec 15, 2010 at 9:47 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Yeah, and more to the point, do I want to finish whatever I was doing in
>>> that window?  Fast-by-default is a nice hammer to swing, but one day
>>> you'll pound your finger.
>
>> I guess.  I've pounded my finger enough time with the current default
>> that I'd be willing to try a different size hammer.  The scenario you
>> describe has yet to occur in 10+ years of using the product, but
>> obviously not everyone's experience will match on this point.
>
> I think the ultimate basis for the way it's set up now is the mantra of
> "be safe by default"; which I believe I've heard you repeating in other
> contexts.  Between that principle and the backwards-compatibility
> hazards, I really don't think there's adequate justification for
> changing this.

Backwards compatibility is, I think, a reasonable argument for
maintaining the current default. However, I don't agree that the
current behavior is safe by default. What often happens is that the
system gets stuck in a state where the existing connections will never
terminate (or not for a long time) but new connections aren't accepted
either. So you're sitting there waiting for the database to shut down
- which it never does - meanwhile, half the people hitting your web
site are getting DOS'd.

Certainly, if you have an environment where people are mostly logging
into the database directly (not through a connection pooler) and they
do a few important queries and then disconnect, smart is a better
default. But if you have an environment where (for whatever reason)
long-lasting connections are common, smart is worse than useless.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Excerpts from Robert Haas's message of mié dic 15 12:03:06 -0300 2010:

> Certainly, if you have an environment where people are mostly logging
> into the database directly (not through a connection pooler) and they
> do a few important queries and then disconnect, smart is a better
> default. But if you have an environment where (for whatever reason)
> long-lasting connections are common, smart is worse than useless.

It occurs to me that we may need a new mode, which disconnects sessions
that are not in a transaction (or as soon as they are) but leaves
in-progress transactions alone; this could be the new default. Of
course, this is much more difficult to implement than the current modes.

--
Álvaro Herrera <alvherre(at)commandprompt(dot)com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
> Excerpts from Robert Haas's message of mié dic 15 12:03:06 -0300 2010:
>
>> Certainly, if you have an environment where people are mostly logging
>> into the database directly (not through a connection pooler) and they
>> do a few important queries and then disconnect, smart is a better
>> default.  But if you have an environment where (for whatever reason)
>> long-lasting connections are common, smart is worse than useless.
>
> It occurs to me that we may need a new mode, which disconnects sessions
> that are not in a transaction (or as soon as they are) but leaves
> in-progress transactions alone; this could be the new default.  Of
> course, this is much more difficult to implement than the current modes.

That would probably be handy, though I think for my use cases fast
would still be better, or smart with a 30-second timeout.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Dec 15, 2010, at 9:11 AM, Alvaro Herrera wrote:
> Excerpts from Robert Haas's message of mié dic 15 12:03:06 -0300 2010:
>
>> Certainly, if you have an environment where people are mostly logging
>> into the database directly (not through a connection pooler) and they
>> do a few important queries and then disconnect, smart is a better
>> default. But if you have an environment where (for whatever reason)
>> long-lasting connections are common, smart is worse than useless.
>
> It occurs to me that we may need a new mode, which disconnects sessions
> that are not in a transaction (or as soon as they are) but leaves
> in-progress transactions alone; this could be the new default. Of
> course, this is much more difficult to implement than the current modes.

+1; that would certainly be useful for us.
--
Jim C. Nasby, Database Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net

On Wed, Dec 15, 2010 at 15:47, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Wed, Dec 15, 2010 at 9:39 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Really?  Personally I'm quite happy with that default.
>
>> Why?  It seems to me that just leads to, oh, gee, the database isn't
>> shutting down, where's the window where I failed to exit a session?
>
> Yeah, and more to the point, do I want to finish whatever I was doing in
> that window?  Fast-by-default is a nice hammer to swing, but one day
> you'll pound your finger.

The whole question "whatever I was doing in that window" indicates a
very limited deployment.

In most production deployments, that would mean different machiens,
and many different people... As a DBA, I certainly don't want to have
to wait around for everybody in my organization to get back from lunch
and close their clients..

In reality, more often than not I see the default shutdown turn into a
very efficient DOS - nobody can do anything in the database ,and a
restart (which is the usual case really - most people don't shut down
their db, they restart it - or shutdown/upgrade/start or something
like that) that could go in seconds turns into minutes or longer.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Wed, Dec 15, 2010 at 16:11, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
> Excerpts from Robert Haas's message of mié dic 15 12:03:06 -0300 2010:
>
>> Certainly, if you have an environment where people are mostly logging
>> into the database directly (not through a connection pooler) and they
>> do a few important queries and then disconnect, smart is a better
>> default.  But if you have an environment where (for whatever reason)
>> long-lasting connections are common, smart is worse than useless.
>
> It occurs to me that we may need a new mode, which disconnects sessions
> that are not in a transaction (or as soon as they are) but leaves
> in-progress transactions alone; this could be the new default.  Of
> course, this is much more difficult to implement than the current modes.

Now that, however, would actually be a useful behavior... Where's your patch? ;)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 12/15/2010 03:47 PM, Tom Lane wrote:
> Yeah, and more to the point, do I want to finish whatever I was doing in
> that window? Fast-by-default is a nice hammer to swing, but one day
> you'll pound your finger.

Magnus pointed out that most distributions already use fast shutdown.
So it seems most people are used to that hammer, no?

Regards

Markus Wanner

On Thu, Dec 16, 2010 at 19:26, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> On 12/15/2010 03:47 PM, Tom Lane wrote:
>> Yeah, and more to the point, do I want to finish whatever I was doing in
>> that window?  Fast-by-default is a nice hammer to swing, but one day
>> you'll pound your finger.
>
> Magnus pointed out that most distributions already use fast shutdown.
> So it seems most people are used to that hammer, no?

Typical users might not use "pg_ctl stop" directly. If they use "service"
or init.d commands, the default shutdown modes are probably fast mode.
It's also true in the service control on MS Windows.

--
Itagaki Takahiro

On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
> It occurs to me that we may need a new mode, which disconnects sessions
> that are not in a transaction (or as soon as they are) but leaves
> in-progress transactions alone; this could be the new default.  Of
> course, this is much more difficult to implement than the current modes.

I like this idea, if it's feasible. Might I also suggest that the
smart-mode shutdown give a HINT to the user that he can forcibly kill
off existing sessions using -m fast. Right now, we show something
like this:

$ pg_ctl -D PGDATA stop
waiting for server to shut down....
........................................................... failed
pg_ctl: server does not shut down

And it's not immediately obvious to the user why the server didn't
shut down, or how to fix things.

Josh

Josh Kupershmidt wrote:
> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera
> <alvherre(at)commandprompt(dot)com> wrote:
> > It occurs to me that we may need a new mode, which disconnects sessions
> > that are not in a transaction (or as soon as they are) but leaves
> > in-progress transactions alone; this could be the new default. ?Of
> > course, this is much more difficult to implement than the current modes.
>
> I like this idea, if it's feasible. Might I also suggest that the
> smart-mode shutdown give a HINT to the user that he can forcibly kill
> off existing sessions using -m fast. Right now, we show something
> like this:
>
> $ pg_ctl -D PGDATA stop
> waiting for server to shut down....
> ........................................................... failed
> pg_ctl: server does not shut down
>
> And it's not immediately obvious to the user why the server didn't
> shut down, or how to fix things.

I have applied the attached patch to mention -m fast when a smart
shutdown or restart fails.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +