Re: [COMMITTERS] pgsql: Reduce the chatter to the log when starting a standby server.

Log Message:
-----------
Reduce the chatter to the log when starting a standby server. Don't
echo all the recovery.conf options. Don't emit the "initializing
recovery connections" message, which doesn't mean anything to a user.
Remove the "starting archive recovery" message and replace the
"automatic recovery in progress" message with a more informative message
saying whether the server is doing PITR, normal archive recovery, or
standby mode.

Modified Files:
--------------
pgsql/src/backend/access/transam:
xlog.c (r1.372 -> r1.373)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xlog.c?r1=1.372&r2=1.373)

On Fri, 2010-02-12 at 09:49 +0000, Heikki Linnakangas wrote:
> Log Message:
> -----------
> Reduce the chatter to the log when starting a standby server. Don't
> echo all the recovery.conf options. Don't emit the "initializing
> recovery connections" message, which doesn't mean anything to a user.
> Remove the "starting archive recovery" message and replace the
> "automatic recovery in progress" message with a more informative message
> saying whether the server is doing PITR, normal archive recovery, or
> standby mode.

Not happy with these changes without discussion.

* "entering standby mode" isn't a more informative message. Two people
have already said on-list that "standby mode" name might need to be
changed. More informative, for me, would be something like "entering
streaming replication mode" and having a parameter called replication
would also make that clearer.

* If you change the HS startup messages you need to change the docs also

--
Simon Riggs www.2ndQuadrant.com

Simon Riggs wrote:
> * "entering standby mode" isn't a more informative message. Two people
> have already said on-list that "standby mode" name might need to be
> changed.

Well, I'm all ears for better suggestions.

> More informative, for me, would be something like "entering
> streaming replication mode" and having a parameter called replication
> would also make that clearer.

That doesn't accurately describe what the standby_mode setting does. It
doesn't imply streaming replication. It means that the server doesn't
end recovery when it reaches end of WAL, but keeps trying.

> * If you change the HS startup messages you need to change the docs also

Thanks, fixed.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Fri, Feb 12, 2010 at 9:55 AM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> That doesn't accurately describe what the standby_mode setting does. It
> doesn't imply streaming replication. It means that the server doesn't
> end recovery when it reaches end of WAL, but keeps trying.

I think I'm going to add my name to the list of people who are unhappy
with standby_mode as a parameter name. I really like the design of
the feature, but the name is just not that clear. You can't read that
parameter name and immediately know what it's trying to do.
Furthermore, if you're wanting to use pg_standby, you might be
forgiven for thinking that you should set standby_mode = on; but in
fact that's exactly the wrong thing to do.

One possibility that occurs to me is that we could call it something
like integrated_standby; but I'm not attached to that.

...Robert

Robert Haas wrote:
> Furthermore, if you're wanting to use pg_standby, you might be
> forgiven for thinking that you should set standby_mode = on; but in
> fact that's exactly the wrong thing to do.

Yeah, I think that's the main weakness of the name "standby_mode". It's
pretty descriptive otherwise, we call that mode of operation "standby"
everywhere, and always have.

I'm not sure I dare to say this out loud after Simon's previous
outburst, but removing or renaming pg_standby would help with that...

> One possibility that occurs to me is that we could call it something
> like integrated_standby; but I'm not attached to that.

Anything with the word 'standby' in it suffers from the same problem,
maybe not as badly but still.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Fri, Feb 12, 2010 at 10:27 AM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> Robert Haas wrote:
>> Furthermore, if you're wanting to use pg_standby, you might be
>> forgiven for thinking that you should set standby_mode = on; but in
>> fact that's exactly the wrong thing to do.
>
> Yeah, I think that's the main weakness of the name "standby_mode". It's
> pretty descriptive otherwise, we call that mode of operation "standby"
> everywhere, and always have.
>
> I'm not sure I dare to say this out loud after Simon's previous
> outburst, but removing or renaming pg_standby would help with that...

Not really. People aren't going to forget about it just because we
remove it from CVS HEAD.

>> One possibility that occurs to me is that we could call it something
>> like integrated_standby; but I'm not attached to that.
>
> Anything with the word 'standby' in it suffers from the same problem,
> maybe not as badly but still.

Well, let's come up with something else then.

...Robert

On Sat, Feb 13, 2010 at 12:28 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> Well, let's come up with something else then.

continuous_recovery ?

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

On 12/02/10 15:37, Fujii Masao wrote:
> On Sat, Feb 13, 2010 at 12:28 AM, Robert Haas<robertmhaas(at)gmail(dot)com> wrote:
>> Well, let's come up with something else then.
>
> continuous_recovery ?

One problem with the otherwise entirely wonderful HS/SR pairing is the
whole business of the config parameters. They feel too bottom-up.
Individually, each one makes sense but if you look at them on a page
they don't say master/slave replication to me.

What about something like:

# Primary
archive_mode = producer
archive_producer_command = 'cp "%p" .../"%f"'
max_consumers = 5

# Standby
archive_mode = producer, consumer
archive_producer_command = 'cp "%p" .../"%f"'
archive_consumer_command = 'cp "%p" .../"%f"'
consume_from = 'host=... user=...'

Three other points that struck me:
1. Why have a separate recovery.conf file rather than just put the
commands inline? We can use the include directive to have them in a
separate file if required.
2. Why have a finish.replication file, rather than "SELECT
pg_finish_replication()"?

--
Richard Huxton
Archonet Ltd

Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
> Robert Haas wrote:
>> Furthermore, if you're wanting to use pg_standby, you might be
>> forgiven for thinking that you should set standby_mode = on; but in
>> fact that's exactly the wrong thing to do.
>
> Yeah, I think that's the main weakness of the name "standby_mode". It's
> pretty descriptive otherwise, we call that mode of operation "standby"
> everywhere, and always have.
>
> I'm not sure I dare to say this out loud after Simon's previous
> outburst, but removing or renaming pg_standby would help with that...

Well seen from here it's quite logical: when replaying WALs, you are
either in recovery mode and the server gets back as soon as possible, or
you are setting up a standby server, which will keep recovering until
told to stop doing so.

Now you have 2 main options for keeping your server in standby mode,
either the integrated one (standby_mode = on) or another one. If you
choose to have your standby state managed by an external tool, of course
the first thing to do is tell the server not to maintain itself the
state, so you switch standby_mode to off.

Then you can either use the included contrib pg_standby to achieve the
result, or some other solution, such as Skytools and walmgr.py or CMD
pitrtools.

The fact that the parameter and the external script share the name is a
hint that they're competing for solving the same problem (in different
ways).

Regards,
--
dim

On Sat, Feb 13, 2010 at 4:33 AM, Richard Huxton <dev(at)archonet(dot)com> wrote:
> 2. Why have a finish.replication file, rather than "SELECT
> pg_finish_replication()"?

I thought that the trigger file method was more easy-to-use for the
existing users since it had been used in pg_standby for a long time
so far. So I used it. But I agree that activating the standby via
SQL would be useful for some cases.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Fujii Masao wrote:
> On Sat, Feb 13, 2010 at 4:33 AM, Richard Huxton <dev(at)archonet(dot)com> wrote:
>> 2. Why have a finish.replication file, rather than "SELECT
>> pg_finish_replication()"?
>
> I thought that the trigger file method was more easy-to-use for the
> existing users since it had been used in pg_standby for a long time
> so far. So I used it.

Also, you might not want to open the standby for connections (ie.
recovery_connections='off'). And even if you wanted, it might not open
because of some of the corner-case limitations in Hot Standby, so you
wouldn't be able to failover.

> But I agree that activating the standby via
> SQL would be useful for some cases.

Yeah.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com