Hot standby v5 patch - restarted replica changes to warm standby mode

While doing some tests yesterday I ran into the situation where the
standby database would appear to go back into 'warm' mode after it was
restarted. The set of steps to reproduce the behaviour is:

1/ Setup master and replica with replica using pg_standby
2/ Initialize pgbench schema with size 100 in database 'postgres'
3/ Connect to replica, then disconnect (this step is not necessary I
*think* - just for checking that connection works at this point!)
4/ Shutdown and restart the replica - there is no "database has now
reached consistent state" message in the log, and you cannot connect

Again this is head from 2nd Nov with v5 patch applied on Freebsd
7.1-Prerelease.

The log fragment is:

LOG: restored log file "000000010000000000000068" from archive
DEBUG: executing restore command "pg_standby -l -d -s 2 -t
/tmp/pgsql.trigger.5439 /data0/pgarchive/8.4 000000010000000000000069
pg_xlog/RECOVERYXLOG 000000010000000000000060 2>>standby.log"
DEBUG: forked new backend, pid=2981 socket=7
FATAL: the database system is starting up
DEBUG: proc_exit(1)
DEBUG: shmem_exit(1)
DEBUG: exit(1)
DEBUG: reaping dead processes
DEBUG: server process (PID 2981) exited with exit code 1

regards

Mark

On Tue, 2008-11-04 at 18:33 +1300, Mark Kirkwood wrote:
> While doing some tests yesterday I ran into the situation where the
> standby database would appear to go back into 'warm' mode after it was
> restarted. The set of steps to reproduce the behaviour is:
>
> 1/ Setup master and replica with replica using pg_standby
> 2/ Initialize pgbench schema with size 100 in database 'postgres'
> 3/ Connect to replica, then disconnect (this step is not necessary I
> *think* - just for checking that connection works at this point!)
> 4/ Shutdown and restart the replica - there is no "database has now
> reached consistent state" message in the log, and you cannot connect

How did you shutdown the database? Fast? Immediate mode acts just as it
does on an unpatched server.

Can you give more details of exactly what you did? Thanks. Not saying
there isn't a problem, just don't understand what happened.

Not being able to connect after a restart is a design feature, to
protect you from running potentially invalid queries.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support

Simon Riggs wrote:
> On Tue, 2008-11-04 at 18:33 +1300, Mark Kirkwood wrote:
>
>> While doing some tests yesterday I ran into the situation where the
>> standby database would appear to go back into 'warm' mode after it was
>> restarted. The set of steps to reproduce the behaviour is:
>>
>> 1/ Setup master and replica with replica using pg_standby
>> 2/ Initialize pgbench schema with size 100 in database 'postgres'
>> 3/ Connect to replica, then disconnect (this step is not necessary I
>> *think* - just for checking that connection works at this point!)
>> 4/ Shutdown and restart the replica - there is no "database has now
>> reached consistent state" message in the log, and you cannot connect
>>
>
> How did you shutdown the database? Fast? Immediate mode acts just as it
> does on an unpatched server.
>
> Can you give more details of exactly what you did? Thanks. Not saying
> there isn't a problem, just don't understand what happened.
>
> Not being able to connect after a restart is a design feature, to
> protect you from running potentially invalid queries.
>
>
Yeah - I was doing it wrong (using immediate). However retesting with
'fast' gets the same result on this platform (Freebsd 7.1). However on
Linux (Ubuntu 8.04) 'fast' shutdown and restart work fine - somewhat
puzzling - I'll try a fresh checkout on the Freebsd boxes, as there may
be something rotten with the src tree I'm using there...

I must may, this is the coolest feature - (from my point of view) the
simplest way to do replication with the minimum of fuss - 3 config
parameters! (archive_mode,archive_command and restore_command). Really
nice addition for 8.4!

regards

Mark

I wrote:
>
> Yeah - I was doing it wrong (using immediate). However retesting with
> 'fast' gets the same result on this platform (Freebsd 7.1). However on
> Linux (Ubuntu 8.04) 'fast' shutdown and restart work fine - somewhat
> puzzling - I'll try a fresh checkout on the Freebsd boxes, as there
> may be something rotten with the src tree I'm using there...
>
Scratch that - I was just being too impatient - after a few more minutes
the replica on Freebsd was accessible ok. So this one is entirely user
error!

regards

Mark

On Wed, 2008-11-05 at 21:05 +1300, Mark Kirkwood wrote:

> So this one is entirely user error!

No worries. I'd rather have false positives, since can often reveal some
usability problem. I think we're OK here for now though.

Thanks very much for testing.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support