Re: How to know killed by pg_terminate_backend

Hi,

If a backend killed by pg_terminate_backend(), the backend returns
57P01 which is identical to the one when it's killed by postmaster.

Problem is, pgpool-II needs to trigger failover if postmaster goes
down because apparently pgpool-II cannot use the PostgreSQL server
anymore.

On the otherhand, pg_terminate_backend() just terminates a backend. So
triggering failover is overkill.

Maybe we could make PostgreSQL a little bit smarter so that it returns
a different code than 57P01 when killed by pg_terminate_backend().

Comments?
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

Tatsuo Ishii wrote:
> If a backend killed by pg_terminate_backend(), the backend returns
> 57P01 which is identical to the one when it's killed by postmaster.
>
> Problem is, pgpool-II needs to trigger failover if postmaster goes
> down because apparently pgpool-II cannot use the PostgreSQL server
> anymore.
>
> On the otherhand, pg_terminate_backend() just terminates a backend. So
> triggering failover is overkill.
>
> Maybe we could make PostgreSQL a little bit smarter so that it returns
> a different code than 57P01 when killed by pg_terminate_backend().

Seems reasonable. Does the victim backend currently know why it has been
killed?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

> > Maybe we could make PostgreSQL a little bit smarter so that it returns
> > a different code than 57P01 when killed by pg_terminate_backend().
>
> Seems reasonable. Does the victim backend currently know why it has been
> killed?

I don't think so.

One idea is postmaster sets a flag in the shared memory area
indicating it rceived SIGTERM before forwarding the signal to
backends.

Backend check the flag and if it's not set, it knows that the signal
has been sent by pg_terminate_backend(), not postmaster.

What about new error code:

#define ERRCODE_BACKEND_STOP_REQUEST MAKE_SQLSTATE('5','7', 'P','0','4')
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

On Thu, May 13, 2010 at 8:20 PM, Tatsuo Ishii <ishii(at)postgresql(dot)org> wrote:
>> > Maybe we could make PostgreSQL a little bit smarter so that it returns
>> > a different code than 57P01 when killed by pg_terminate_backend().
>>
>> Seems reasonable. Does the victim backend currently know why it has been
>> killed?
>
> I don't think so.
>
> One idea is postmaster sets a flag in the shared memory area
> indicating it rceived SIGTERM before forwarding the signal to
> backends.
>
> Backend check the flag and if it's not set, it knows that the signal
> has been sent by pg_terminate_backend(), not postmaster.

Or it could also be sent by some other user process, like the user
running "kill" from the shell.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

> >> Seems reasonable. Does the victim backend currently know why it has been
> >> killed?
> >
> > I don't think so.
> >
> > One idea is postmaster sets a flag in the shared memory area
> > indicating it rceived SIGTERM before forwarding the signal to
> > backends.
> >
> > Backend check the flag and if it's not set, it knows that the signal
> > has been sent by pg_terminate_backend(), not postmaster.
>
> Or it could also be sent by some other user process, like the user
> running "kill" from the shell.

No problem (at least for pgpool-II).

If the flag is not set, postgres returns the same code as the one
killed by pg_terminate_backend(). The point is, backend is killed by
postmaster or not. Because if backend was killed by postmaster,
pgpool-II should not expect the PostgreSQL server is usable since
postmaster decided to shutdown.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

>> >> Seems reasonable. Does the victim backend currently know why it has been
>> >> killed?
>> >
>> > I don't think so.
>> >
>> > One idea is postmaster sets a flag in the shared memory area
>> > indicating it rceived SIGTERM before forwarding the signal to
>> > backends.
>> >
>> > Backend check the flag and if it's not set, it knows that the signal
>> > has been sent by pg_terminate_backend(), not postmaster.
>>
>> Or it could also be sent by some other user process, like the user
>> running "kill" from the shell.
>
> No problem (at least for pgpool-II).
>
> If the flag is not set, postgres returns the same code as the one
> killed by pg_terminate_backend(). The point is, backend is killed by
> postmaster or not. Because if backend was killed by postmaster,
> pgpool-II should not expect the PostgreSQL server is usable since
> postmaster decided to shutdown.

Here is the patch to implement the feature.

1) pg_terminate_backend() sends SIGUSR1 signal rather than SIGTERM to
the target backend.
2) The infrastructure used for message passing is
storage/ipc/procsignal.c The new message type for ProcSignalReason
is "PROCSIG_TERMNINATE_BACKEND_INTERRUPT"
3) I assign new error code 57P04 which is returned from the backend
killed by pg_terminate_backend().

#define ERRCODE_TERMINATE_BACKEND MAKE_SQLSTATE('5','7', 'P','0','4')

Comments are welcome.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

>> >> Seems reasonable. Does the victim backend currently know why it has been
>> >> killed?
>> >
>> > I don't think so.
>> >
>> > One idea is postmaster sets a flag in the shared memory area
>> > indicating it rceived SIGTERM before forwarding the signal to
>> > backends.
>> >
>> > Backend check the flag and if it's not set, it knows that the signal
>> > has been sent by pg_terminate_backend(), not postmaster.
>>
>> Or it could also be sent by some other user process, like the user
>> running "kill" from the shell.
>
> No problem (at least for pgpool-II).
>
> If the flag is not set, postgres returns the same code as the one
> killed by pg_terminate_backend(). The point is, backend is killed by
> postmaster or not. Because if backend was killed by postmaster,
> pgpool-II should not expect the PostgreSQL server is usable since
> postmaster decided to shutdown.

Here is the patch to implement the feature.

1) pg_terminate_backend() sends SIGUSR1 signal rather than SIGTERM to
the target backend.
2) The infrastructure used for message passing is
storage/ipc/procsignal.c The new message type for ProcSignalReason
is "PROCSIG_TERMNINATE_BACKEND_INTERRUPT"
3) I assign new error code 57P04 which is returned from the backend
killed by pg_terminate_backend().

#define ERRCODE_TERMINATE_BACKEND MAKE_SQLSTATE('5','7', 'P','0','4')

Comments are welcome.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

Tatsuo Ishii <ishii(at)postgresql(dot)org> writes:
> Comments are welcome.

This is a bad idea. It makes an already-poorly-tested code path
significantly more fragile, in return for nothing of value.

regards, tom lane

> Tatsuo Ishii <ishii(at)postgresql(dot)org> writes:
>> Comments are welcome.
>
> This is a bad idea. It makes an already-poorly-tested code path
> significantly more fragile, in return for nothing of value.

Are you saying that procsignal.c is the already-poorly-tested one? If
so, why?

As for "value", I have already explained why we need this in the
upthread.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

> Here is the patch to implement the feature.
>
> 1) pg_terminate_backend() sends SIGUSR1 signal rather than SIGTERM to
> the target backend.
> 2) The infrastructure used for message passing is
> storage/ipc/procsignal.c The new message type for ProcSignalReason
> is "PROCSIG_TERMNINATE_BACKEND_INTERRUPT"
> 3) I assign new error code 57P04 which is returned from the backend
> killed by pg_terminate_backend().
>
> #define ERRCODE_TERMINATE_BACKEND MAKE_SQLSTATE('5','7', 'P','0','4')
>
> Comments are welcome.

Anyone has better idea? Tom dislikes my patch but I don't know how to
deal with it.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

On Fri, Jan 21, 2011 at 13:56, Tatsuo Ishii <ishii(at)postgresql(dot)org> wrote:
>> Here is the patch to implement the feature.
>>
>> 1) pg_terminate_backend() sends SIGUSR1 signal rather than SIGTERM to
>>    the target backend.
>> 2) The infrastructure used for message passing is
>>    storage/ipc/procsignal.c The new message type for ProcSignalReason
>>    is "PROCSIG_TERMNINATE_BACKEND_INTERRUPT"
>>  3) I assign new error code 57P04 which is returned from the backend
>>       killed by pg_terminate_backend().
>>
>> #define ERRCODE_TERMINATE_BACKEND                     MAKE_SQLSTATE('5','7', 'P','0','4')
>
> Anyone has better idea? Tom dislikes my patch but I don't know how to
> deal with it.

There was another design in the past discussion:
>> One idea is postmaster sets a flag in the shared memory area
>> indicating it rceived SIGTERM before forwarding the signal to
>> backends.

Is it enough for your purpose and do we think it is more robust way?

--
Itagaki Takahiro

Itagaki Takahiro <itagaki(dot)takahiro(at)gmail(dot)com> writes:
> On Fri, Jan 21, 2011 at 13:56, Tatsuo Ishii <ishii(at)postgresql(dot)org> wrote:
>> Anyone has better idea? Tom dislikes my patch but I don't know how to
>> deal with it.

> There was another design in the past discussion:
> One idea is postmaster sets a flag in the shared memory area
> indicating it rceived SIGTERM before forwarding the signal to
> backends.

> Is it enough for your purpose and do we think it is more robust way?

To put this as briefly as possible: I don't want to add even one line of
code to distinguish pg_terminate_backend from database-wide shutdown.
That function should be a last-ditch tool, not something used on a daily
basis. So I disagree with the premise as much as with any particular
implementation.

regards, tom lane

On Fri, Jan 21, 2011 at 10:35 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Itagaki Takahiro <itagaki(dot)takahiro(at)gmail(dot)com> writes:
>> On Fri, Jan 21, 2011 at 13:56, Tatsuo Ishii <ishii(at)postgresql(dot)org> wrote:
>>> Anyone has better idea? Tom dislikes my patch but I don't know how to
>>> deal with it.
>
>> There was another design in the past discussion:
>> One idea is postmaster sets a flag in the shared memory area
>> indicating it rceived SIGTERM before forwarding the signal to
>> backends.
>
>> Is it enough for your purpose and do we think it is more robust way?
>
> To put this as briefly as possible: I don't want to add even one line of
> code to distinguish pg_terminate_backend from database-wide shutdown.
> That function should be a last-ditch tool, not something used on a daily
> basis.  So I disagree with the premise as much as with any particular
> implementation.

Well, that seems awfully unfriendly.

Frequency of use is beside the point - people are trying to write
client applications - like pgpool-II - that understand the behavior of
PG. If we send the same error code in two different situations with
different behaviors, such applications have to do so silly workarounds
to figure out what really happened.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company