Lists: | pgsql-hackerspgsql-patches |
---|
From: | "Simon Riggs" <simon(at)2ndquadrant(dot)com> |
---|---|
To: | <pgsql-patches(at)postgresql(dot)org> |
Subject: | pg_standby |
Date: | 2006-12-14 12:04:10 |
Message-ID: | 1166097851.3733.74.camel@silverbirch.site |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
pg_standby and test framework, in separate .tar files
pg_standby
----------
pg_standby is a production-ready program that can be used to
create a Warm Standby server with PostgreSQL.
The program is designed to be a wait-for restore_command,
required to turn a normal archive recovery into a Warm Standby.
Within the restore_command of the recovery.conf you could
configure pg_standby in the following way:
restore_command = 'pg_standby archiveDir %f %p'
$pg_standby
pg_standby allows Warm Standby servers to be configured
Usage:
pg_standby [OPTION]... [ARCHIVELOCATION] [NEXTWALFILE] [XLOGFILEPATH]
note space between [ARCHIVELOCATION] and
[NEXTWALFILE]
with main intended use via restore_command in the recovery.conf
restore_command = 'pg_standby [OPTION]... [ARCHIVELOCATION] %f %p'
e.g. restore_command = 'pg_standby -m /mnt/server/archiverdir %f %p'
Options:
-d generate lots of debugging output (testing
only)
-m moves file rather than copying from archive
-t [TRIGGERFILE] defines a trigger file to initiate failover
(no default)
-s [SLEEPTIME] number of seconds to wait between file checks
(default=5)
-w [MAXWAITTIME] max number of seconds to wait for a file (0
disables)(default=600)
pg_standby runs standalone and as a restore_command. Tested and working
successfully in both modes.
No signal handling - do we need some?
Works successfully with shutdown of standby server and via trigger file.
test_warm_standby
-----------------
bash script to run two PostgreSQL servers, one Primary, one Standby -
both running on same system. Servers use non-standard port numbers
deliberately, to avoid conflicts with other systems.
Designed to be executed from /usr/local/pgsql, nothing too fancy
File contents:
$ tar tf pg_standby.tar
contrib/pg_standby/
contrib/pg_standby/Makefile
contrib/pg_standby/pg_standby.c
contrib/pg_standby/README.pg_standby
allows make, make install, make distclean
intended for submission to core as a contrib module
$tar tf test_warm_standby.tar
test_warm_standby.primary.postgresql.conf
test_warm_standby.standby.postgresql.conf
test_warm_standby.standby.recovery.conf
test_warm_standby.start.sh
test_warm_standby.stop.sh
needs some discussion, code needs enhancement before commit
maybe implement config changes as edits rather than full scripts
All feedback welcome.
--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com
Attachment | Content-Type | Size |
---|---|---|
pg_standby.tar | application/x-tar | 20.0 KB |
test_warm_standby.tar | application/x-tar | 40.0 KB |
From: | "Simon Riggs" <simon(at)2ndquadrant(dot)com> |
---|---|
To: | <pgsql-patches(at)postgresql(dot)org> |
Cc: | "Doug Knight" <dknight(at)wsi(dot)com> |
Subject: | Re: pg_standby |
Date: | 2006-12-28 19:26:27 |
Message-ID: | 1167333987.3633.169.camel@silverbirch.site |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> pg_standby and test framework, in separate .tar files
New version (v2), following further testing.
Signal handling not included in this version.
--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com
Attachment | Content-Type | Size |
---|---|---|
pg_standby.v2.tar | application/x-tar | 20.0 KB |
From: | "Simon Riggs" <simon(at)2ndquadrant(dot)com> |
---|---|
To: | <pgsql-patches(at)postgresql(dot)org> |
Cc: | "Doug Knight" <dknight(at)wsi(dot)com> |
Subject: | Re: pg_standby |
Date: | 2006-12-28 22:47:23 |
Message-ID: | 1167346044.3633.200.camel@silverbirch.site |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > pg_standby and test framework, in separate .tar files
>
> New version (v2), following further testing.
>
> Signal handling not included in this version.
Signal handling now added, tested and working correctly in version 3,
attached.
pg_standby is an example program for a warm standby script as discussed
on -hackers:
http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
Program looks complete and ready for review, to me.
--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com
Attachment | Content-Type | Size |
---|---|---|
pg_standby.v3.tar | application/x-tar | 20.0 KB |
From: | "Merlin Moncure" <mmoncure(at)gmail(dot)com> |
---|---|
To: | "Simon Riggs" <simon(at)2ndquadrant(dot)com> |
Cc: | pgsql-patches(at)postgresql(dot)org, "Doug Knight" <dknight(at)wsi(dot)com> |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-01-17 15:05:00 |
Message-ID: | b42b73150701170705j76a0344csf4cae66ce8ea4793@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On 12/28/06, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> > On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > > pg_standby and test framework, in separate .tar files
> >
> > New version (v2), following further testing.
> >
> > Signal handling not included in this version.
>
> Signal handling now added, tested and working correctly in version 3,
> attached.
>
> pg_standby is an example program for a warm standby script as discussed
> on -hackers:
> http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
>
> Program looks complete and ready for review, to me.
I double checked and re-ran all my test and confirmed that pg_standby
move (-m) mode is definitely busted in v3 in the sense that a restart
of the standby will not resume recovery and requires a pg_resetxlog to
become operational -- it needs one more WAL file back than the oldest
one available.
I am currently working around this by rotating WAL files a couple of
versions back in the shell script I am using to receive log files via
netcat. move mode is very desirable because it keeps the maintenance
down for the standby system.
merlin
From: | Doug Knight <dknight(at)wsi(dot)com> |
---|---|
To: | Merlin Moncure <mmoncure(at)gmail(dot)com> |
Cc: | Simon Riggs <simon(at)2ndquadrant(dot)com>, pgsql-patches(at)postgresql(dot)org |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-01-17 15:10:00 |
Message-ID: | 1169046600.13711.1.camel@localhost.localdomain |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
I confirm that I am seeing the exact same characteristic. Could you post
your rotating script?
Thanks,
Doug
On Wed, 2007-01-17 at 10:05 -0500, Merlin Moncure wrote:
> On 12/28/06, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> > > On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > > > pg_standby and test framework, in separate .tar files
> > >
> > > New version (v2), following further testing.
> > >
> > > Signal handling not included in this version.
> >
> > Signal handling now added, tested and working correctly in version 3,
> > attached.
> >
> > pg_standby is an example program for a warm standby script as discussed
> > on -hackers:
> > http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
> >
> > Program looks complete and ready for review, to me.
>
> I double checked and re-ran all my test and confirmed that pg_standby
> move (-m) mode is definitely busted in v3 in the sense that a restart
> of the standby will not resume recovery and requires a pg_resetxlog to
> become operational -- it needs one more WAL file back than the oldest
> one available.
>
> I am currently working around this by rotating WAL files a couple of
> versions back in the shell script I am using to receive log files via
> netcat. move mode is very desirable because it keeps the maintenance
> down for the standby system.
>
> merlin
>
From: | "Merlin Moncure" <mmoncure(at)gmail(dot)com> |
---|---|
To: | "Doug Knight" <dknight(at)wsi(dot)com> |
Cc: | "Simon Riggs" <simon(at)2ndquadrant(dot)com>, pgsql-patches(at)postgresql(dot)org |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-01-17 15:31:25 |
Message-ID: | b42b73150701170731o3c266607h5a7a33e31426dd80@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On 1/17/07, Doug Knight <dknight(at)wsi(dot)com> wrote:
> I confirm that I am seeing the exact same characteristic. Could you post
> your rotating script?
note: this is still a work in progress, the crude but effective sleep
5 is due to be replaced with a lock/fifo and there catch_wal.sh needs
to be rewritten a bit. truncate is a C one-liner I wrote which does a
ftruncate.
*** primary: ***
archive_command = '/home/postgres/send_wal.sh %p %f'
*** send_wal.sh: ***
!/bin/bash
echo "archiving: $2" >> ~/send_wal.log
cat $1 <(echo "placeholder") <(echo $2) | nc $STANDBY 1234 && sleep 5
*** secondary: ***
restore_command = 'pg_standby -m -w0 -t/raid/pitr/kill /raid/pitr %f %p'
*** catch_wal.sh ***
!/bin/bash
WALDIR=/raid/pitr
rm -f $WALDIR/*.old
rm -f $WALDIR/*.older
> $WALDIR/tmp.older
> $WALDIR/tmp.old
while true;
do
tmpfile=`mktemp`
nc -l 1234 > $tmpfile || { echo "FATAL: nc listen failed"; exit 1; }
chown postgres:postgres $tmpfile
file_name=`tail -1 $tmpfile`
./truncate $tmpfile 16777216
rm -f $WALDIR/*.older
for i in `ls $WALDIR/*.old`; do mv $i $WALDIR/`basename $i .old`.older; done
mv $tmpfile $WALDIR/$file_name.old
cp --preserve=ownership $WALDIR/$file_name.old $WALDIR/$file_name
echo "LOG: caught file: $file_name"
done
From: | "Merlin Moncure" <mmoncure(at)gmail(dot)com> |
---|---|
To: | "Doug Knight" <dknight(at)wsi(dot)com> |
Cc: | "Simon Riggs" <simon(at)2ndquadrant(dot)com>, pgsql-patches(at)postgresql(dot)org |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-01-17 16:03:24 |
Message-ID: | b42b73150701170803o7e2ecf23k5c988cb1bd23bb02@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On 1/17/07, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> On 1/17/07, Doug Knight <dknight(at)wsi(dot)com> wrote:
> > I confirm that I am seeing the exact same characteristic. Could you post
> > your rotating script?
>
> note: this is still a work in progress, the crude but effective sleep
> 5 is due to be replaced with a lock/fifo and there catch_wal.sh needs
> to be rewritten a bit. truncate is a C one-liner I wrote which does a
> ftruncate.
>
this turned out not to fix the problem...working on it still!
merlin
From: | "Merlin Moncure" <mmoncure(at)gmail(dot)com> |
---|---|
To: | "Simon Riggs" <simon(at)2ndquadrant(dot)com> |
Cc: | "Dave Page" <dpage(at)postgresql(dot)org>, pgsql-patches(at)postgresql(dot)org, "Doug Knight" <dknight(at)wsi(dot)com> |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-01-17 18:50:59 |
Message-ID: | b42b73150701171050r262e63a4w73dad3915fd4f042@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On 1/17/07, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> new v4
>
> Changes
> - removed -m command, design flaw in original spec, use -l instead
> - added -k N command to cleanup archive and leave max N files
> - fflush() points added to allow Windows debug
> - bug fix: when .history file present
> - bug fix: command line switch cleanup
> - readme updated
works fantastic. grazi...i guess my rotation would have worked with
more files but -k is much cleaner.
merlin
From: | "Simon Riggs" <simon(at)2ndquadrant(dot)com> |
---|---|
To: | "Merlin Moncure" <mmoncure(at)gmail(dot)com> |
Cc: | "Dave Page" <dpage(at)postgresql(dot)org>, <pgsql-patches(at)postgresql(dot)org>, "Doug Knight" <dknight(at)wsi(dot)com> |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-01-22 13:06:53 |
Message-ID: | 1169471213.3776.262.camel@silverbirch.site |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On Wed, 2007-01-17 at 16:15 +0000, Simon Riggs wrote:
> On Wed, 2007-01-17 at 10:05 -0500, Merlin Moncure wrote:
> > On 12/28/06, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > > On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> > > > On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > > > > pg_standby and test framework, in separate .tar files
> > > >
> > > > New version (v2), following further testing.
> > > >
> > > > Signal handling not included in this version.
> > >
> > > Signal handling now added, tested and working correctly in version 3,
> > > attached.
> > >
> > > pg_standby is an example program for a warm standby script as discussed
> > > on -hackers:
> > > http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
> > >
> > > Program looks complete and ready for review, to me.
> >
> > I double checked and re-ran all my test and confirmed that pg_standby
> > move (-m) mode is definitely busted in v3 in the sense that a restart
> > of the standby will not resume recovery and requires a pg_resetxlog to
> > become operational -- it needs one more WAL file back than the oldest
> > one available.
>
> new v4
>
> Changes
> - removed -m command, design flaw in original spec, use -l instead
> - added -k N command to cleanup archive and leave max N files
> - fflush() points added to allow Windows debug
> - bug fix: when .history file present
> - bug fix: command line switch cleanup
> - readme updated
new v6 (v5 was Windows dev release)
Changes
- added -r option to specify maxretries
- -l option for Windows Vista (only) using mklink
- Windows examples and docs added to readme
- code restructured to allow more easy customization
- bug fix: -k 0 error fixed
- successful port report from Dave Page on Windows XP
--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com
Attachment | Content-Type | Size |
---|---|---|
pg_standby.v6.tar | application/x-tar | 30.0 KB |
From: | Bruce Momjian <bruce(at)momjian(dot)us> |
---|---|
To: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
Cc: | Merlin Moncure <mmoncure(at)gmail(dot)com>, Dave Page <dpage(at)postgresql(dot)org>, pgsql-patches(at)postgresql(dot)org, Doug Knight <dknight(at)wsi(dot)com> |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-01-31 04:16:28 |
Message-ID: | 200701310416.l0V4GSj19242@momjian.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
Your patch has been added to the PostgreSQL unapplied patches list at:
http://momjian.postgresql.org/cgi-bin/pgpatches
It will be applied as soon as one of the PostgreSQL committers reviews
and approves it.
---------------------------------------------------------------------------
Simon Riggs wrote:
> On Wed, 2007-01-17 at 16:15 +0000, Simon Riggs wrote:
> > On Wed, 2007-01-17 at 10:05 -0500, Merlin Moncure wrote:
> > > On 12/28/06, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > > > On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> > > > > On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > > > > > pg_standby and test framework, in separate .tar files
> > > > >
> > > > > New version (v2), following further testing.
> > > > >
> > > > > Signal handling not included in this version.
> > > >
> > > > Signal handling now added, tested and working correctly in version 3,
> > > > attached.
> > > >
> > > > pg_standby is an example program for a warm standby script as discussed
> > > > on -hackers:
> > > > http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
> > > >
> > > > Program looks complete and ready for review, to me.
> > >
> > > I double checked and re-ran all my test and confirmed that pg_standby
> > > move (-m) mode is definitely busted in v3 in the sense that a restart
> > > of the standby will not resume recovery and requires a pg_resetxlog to
> > > become operational -- it needs one more WAL file back than the oldest
> > > one available.
> >
> > new v4
> >
> > Changes
> > - removed -m command, design flaw in original spec, use -l instead
> > - added -k N command to cleanup archive and leave max N files
> > - fflush() points added to allow Windows debug
> > - bug fix: when .history file present
> > - bug fix: command line switch cleanup
> > - readme updated
>
> new v6 (v5 was Windows dev release)
>
> Changes
>
> - added -r option to specify maxretries
> - -l option for Windows Vista (only) using mklink
> - Windows examples and docs added to readme
> - code restructured to allow more easy customization
> - bug fix: -k 0 error fixed
>
> - successful port report from Dave Page on Windows XP
>
> --
> Simon Riggs
> EnterpriseDB http://www.enterprisedb.com
>
[ Attachment, skipping... ]
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> message can get through to the mailing list cleanly
--
Bruce Momjian bruce(at)momjian(dot)us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
From: | Doug Knight <dknight(at)wsi(dot)com> |
---|---|
To: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
Cc: | pgsql-patches(at)postgresql(dot)org |
Subject: | Re: pg_standby |
Date: | 2007-02-01 20:14:48 |
Message-ID: | 1170360888.10933.50.camel@arc-dknightlx.wsicorp.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
Hi Simon,
Quick question on the -w option; setting it to zero "disables", do you
mean it waits until the file appears or a trigger file appears, or it
just doesn't wait at all?
Doug Knight
WSI Inc
Andover, MA
On Mon, 2007-01-22 at 13:06 +0000, Simon Riggs wrote:
> On Wed, 2007-01-17 at 16:15 +0000, Simon Riggs wrote:
> > On Wed, 2007-01-17 at 10:05 -0500, Merlin Moncure wrote:
> > > On 12/28/06, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > > > On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> > > > > On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > > > > > pg_standby and test framework, in separate .tar files
> > > > >
> > > > > New version (v2), following further testing.
> > > > >
> > > > > Signal handling not included in this version.
> > > >
> > > > Signal handling now added, tested and working correctly in version 3,
> > > > attached.
> > > >
> > > > pg_standby is an example program for a warm standby script as discussed
> > > > on -hackers:
> > > > http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
> > > >
> > > > Program looks complete and ready for review, to me.
> > >
> > > I double checked and re-ran all my test and confirmed that pg_standby
> > > move (-m) mode is definitely busted in v3 in the sense that a restart
> > > of the standby will not resume recovery and requires a pg_resetxlog to
> > > become operational -- it needs one more WAL file back than the oldest
> > > one available.
> >
> > new v4
> >
> > Changes
> > - removed -m command, design flaw in original spec, use -l instead
> > - added -k N command to cleanup archive and leave max N files
> > - fflush() points added to allow Windows debug
> > - bug fix: when .history file present
> > - bug fix: command line switch cleanup
> > - readme updated
>
> new v6 (v5 was Windows dev release)
>
> Changes
>
> - added -r option to specify maxretries
> - -l option for Windows Vista (only) using mklink
> - Windows examples and docs added to readme
> - code restructured to allow more easy customization
> - bug fix: -k 0 error fixed
>
> - successful port report from Dave Page on Windows XP
>
From: | "Simon Riggs" <simon(at)2ndquadrant(dot)com> |
---|---|
To: | "Doug Knight" <dknight(at)wsi(dot)com> |
Cc: | <pgsql-patches(at)postgresql(dot)org> |
Subject: | Re: pg_standby |
Date: | 2007-02-02 14:38:44 |
Message-ID: | 1170427124.3680.233.camel@silverbirch.site |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On Thu, 2007-02-01 at 15:14 -0500, Doug Knight wrote:
> Quick question on the -w option; setting it to zero "disables", do you
> mean it waits until the file appears or a trigger file appears, or it
> just doesn't wait at all?
It means it waits forever, or until a trigger file appears - but a
trigger file is optional, so its possible to create an awkward
situation. I'm not happy with that default, but feedback from Merlin
suggested production problems with people not understanding that.
I'm happy to change to whatever consensus is, so if you think that's
dumb, just shout.
--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com
From: | Doug Knight <dknight(at)wsi(dot)com> |
---|---|
To: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
Cc: | pgsql-patches(at)postgresql(dot)org |
Subject: | Re: pg_standby |
Date: | 2007-02-02 14:49:23 |
Message-ID: | 1170427763.10933.60.camel@arc-dknightlx.wsicorp.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
Not at all, in fact I was planning on using the infinite wait, and using
something like heartbeat to force creation of the trigger file in the
event the primary dies. Thanks Simon!
Doug
On Fri, 2007-02-02 at 14:38 +0000, Simon Riggs wrote:
> On Thu, 2007-02-01 at 15:14 -0500, Doug Knight wrote:
>
> > Quick question on the -w option; setting it to zero "disables", do you
> > mean it waits until the file appears or a trigger file appears, or it
> > just doesn't wait at all?
>
> It means it waits forever, or until a trigger file appears - but a
> trigger file is optional, so its possible to create an awkward
> situation. I'm not happy with that default, but feedback from Merlin
> suggested production problems with people not understanding that.
>
> I'm happy to change to whatever consensus is, so if you think that's
> dumb, just shout.
>
From: | Bruce Momjian <bruce(at)momjian(dot)us> |
---|---|
To: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
Cc: | Merlin Moncure <mmoncure(at)gmail(dot)com>, Dave Page <dpage(at)postgresql(dot)org>, pgsql-patches(at)postgresql(dot)org, Doug Knight <dknight(at)wsi(dot)com> |
Subject: | Re: [pgsql-patches] pg_standby |
Date: | 2007-02-08 15:09:51 |
Message-ID: | 200702081509.l18F9pp01121@momjian.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
Patch applied. Thanks.
---------------------------------------------------------------------------
Simon Riggs wrote:
> On Wed, 2007-01-17 at 16:15 +0000, Simon Riggs wrote:
> > On Wed, 2007-01-17 at 10:05 -0500, Merlin Moncure wrote:
> > > On 12/28/06, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > > > On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> > > > > On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > > > > > pg_standby and test framework, in separate .tar files
> > > > >
> > > > > New version (v2), following further testing.
> > > > >
> > > > > Signal handling not included in this version.
> > > >
> > > > Signal handling now added, tested and working correctly in version 3,
> > > > attached.
> > > >
> > > > pg_standby is an example program for a warm standby script as discussed
> > > > on -hackers:
> > > > http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
> > > >
> > > > Program looks complete and ready for review, to me.
> > >
> > > I double checked and re-ran all my test and confirmed that pg_standby
> > > move (-m) mode is definitely busted in v3 in the sense that a restart
> > > of the standby will not resume recovery and requires a pg_resetxlog to
> > > become operational -- it needs one more WAL file back than the oldest
> > > one available.
> >
> > new v4
> >
> > Changes
> > - removed -m command, design flaw in original spec, use -l instead
> > - added -k N command to cleanup archive and leave max N files
> > - fflush() points added to allow Windows debug
> > - bug fix: when .history file present
> > - bug fix: command line switch cleanup
> > - readme updated
>
> new v6 (v5 was Windows dev release)
>
> Changes
>
> - added -r option to specify maxretries
> - -l option for Windows Vista (only) using mklink
> - Windows examples and docs added to readme
> - code restructured to allow more easy customization
> - bug fix: -k 0 error fixed
>
> - successful port report from Dave Page on Windows XP
>
> --
> Simon Riggs
> EnterpriseDB http://www.enterprisedb.com
>
[ Attachment, skipping... ]
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> message can get through to the mailing list cleanly
--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
From: | Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> |
---|---|
To: | pgsql-patches(at)postgresql(dot)org |
Cc: | Bruce Momjian <bruce(at)momjian(dot)us>, Simon Riggs <simon(at)2ndquadrant(dot)com> |
Subject: | Re: [pgsql-patches] pg_standby |
Date: | 2007-02-16 22:44:45 |
Message-ID: | 200702161744.45801.xzilla@users.sourceforge.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
At the risk of starting trouble, is there some reason this was added to
contrib and not put on pgfoundry ?
On Thursday 08 February 2007 10:09, Bruce Momjian wrote:
> Patch applied. Thanks.
>
> ---------------------------------------------------------------------------
>
> Simon Riggs wrote:
> > On Wed, 2007-01-17 at 16:15 +0000, Simon Riggs wrote:
> > > On Wed, 2007-01-17 at 10:05 -0500, Merlin Moncure wrote:
> > > > On 12/28/06, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > > > > On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> > > > > > On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > > > > > > pg_standby and test framework, in separate .tar files
> > > > > >
> > > > > > New version (v2), following further testing.
> > > > > >
> > > > > > Signal handling not included in this version.
> > > > >
> > > > > Signal handling now added, tested and working correctly in version
> > > > > 3, attached.
> > > > >
> > > > > pg_standby is an example program for a warm standby script as
> > > > > discussed on -hackers:
> > > > > http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
> > > > >
> > > > > Program looks complete and ready for review, to me.
> > > >
> > > > I double checked and re-ran all my test and confirmed that pg_standby
> > > > move (-m) mode is definitely busted in v3 in the sense that a restart
> > > > of the standby will not resume recovery and requires a pg_resetxlog
> > > > to become operational -- it needs one more WAL file back than the
> > > > oldest one available.
> > >
> > > new v4
> > >
> > > Changes
> > > - removed -m command, design flaw in original spec, use -l instead
> > > - added -k N command to cleanup archive and leave max N files
> > > - fflush() points added to allow Windows debug
> > > - bug fix: when .history file present
> > > - bug fix: command line switch cleanup
> > > - readme updated
> >
> > new v6 (v5 was Windows dev release)
> >
> > Changes
> >
> > - added -r option to specify maxretries
> > - -l option for Windows Vista (only) using mklink
> > - Windows examples and docs added to readme
> > - code restructured to allow more easy customization
> > - bug fix: -k 0 error fixed
> >
> > - successful port report from Dave Page on Windows XP
> >
> > --
> > Simon Riggs
> > EnterpriseDB http://www.enterprisedb.com
>
> [ Attachment, skipping... ]
>
> > ---------------------------(end of broadcast)---------------------------
> > TIP 1: if posting/reading through Usenet, please send an appropriate
> > subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> > message can get through to the mailing list cleanly
--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
From: | Bruce Momjian <bruce(at)momjian(dot)us> |
---|---|
To: | Robert Treat <xzilla(at)users(dot)sourceforge(dot)net> |
Cc: | pgsql-patches(at)postgresql(dot)org, Simon Riggs <simon(at)2ndquadrant(dot)com> |
Subject: | Re: [pgsql-patches] pg_standby |
Date: | 2007-02-16 23:01:56 |
Message-ID: | 200702162301.l1GN1uO17825@momjian.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
Robert Treat wrote:
> At the risk of starting trouble, is there some reason this was added to
> contrib and not put on pgfoundry ?
I thought the idea was that it was integral to using PITR, but might
change so it was put in /contrib.
>
> On Thursday 08 February 2007 10:09, Bruce Momjian wrote:
> > Patch applied. Thanks.
> >
> > ---------------------------------------------------------------------------
> >
> > Simon Riggs wrote:
> > > On Wed, 2007-01-17 at 16:15 +0000, Simon Riggs wrote:
> > > > On Wed, 2007-01-17 at 10:05 -0500, Merlin Moncure wrote:
> > > > > On 12/28/06, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> > > > > > On Thu, 2006-12-28 at 19:26 +0000, Simon Riggs wrote:
> > > > > > > On Thu, 2006-12-14 at 12:04 +0000, Simon Riggs wrote:
> > > > > > > > pg_standby and test framework, in separate .tar files
> > > > > > >
> > > > > > > New version (v2), following further testing.
> > > > > > >
> > > > > > > Signal handling not included in this version.
> > > > > >
> > > > > > Signal handling now added, tested and working correctly in version
> > > > > > 3, attached.
> > > > > >
> > > > > > pg_standby is an example program for a warm standby script as
> > > > > > discussed on -hackers:
> > > > > > http://archives.postgresql.org/pgsql-hackers/2006-08/msg00407.php
> > > > > >
> > > > > > Program looks complete and ready for review, to me.
> > > > >
> > > > > I double checked and re-ran all my test and confirmed that pg_standby
> > > > > move (-m) mode is definitely busted in v3 in the sense that a restart
> > > > > of the standby will not resume recovery and requires a pg_resetxlog
> > > > > to become operational -- it needs one more WAL file back than the
> > > > > oldest one available.
> > > >
> > > > new v4
> > > >
> > > > Changes
> > > > - removed -m command, design flaw in original spec, use -l instead
> > > > - added -k N command to cleanup archive and leave max N files
> > > > - fflush() points added to allow Windows debug
> > > > - bug fix: when .history file present
> > > > - bug fix: command line switch cleanup
> > > > - readme updated
> > >
> > > new v6 (v5 was Windows dev release)
> > >
> > > Changes
> > >
> > > - added -r option to specify maxretries
> > > - -l option for Windows Vista (only) using mklink
> > > - Windows examples and docs added to readme
> > > - code restructured to allow more easy customization
> > > - bug fix: -k 0 error fixed
> > >
> > > - successful port report from Dave Page on Windows XP
> > >
> > > --
> > > Simon Riggs
> > > EnterpriseDB http://www.enterprisedb.com
> >
> > [ Attachment, skipping... ]
> >
> > > ---------------------------(end of broadcast)---------------------------
> > > TIP 1: if posting/reading through Usenet, please send an appropriate
> > > subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> > > message can get through to the mailing list cleanly
>
> --
> Robert Treat
> Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
From: | Doug Knight <dknight(at)wsi(dot)com> |
---|---|
To: | Simon Riggs <simon(at)2ndquadrant(dot)com>, pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-03-08 18:29:50 |
Message-ID: | 1173378590.4829.48.camel@arc-dknightlx.wsicorp.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
Hi Simon,
I would preserve the existing trigger function as little t "-t", and
maybe implement a catchup trigger function as big t "-T"? Set it up so
that if the first attempt to find the WAL file postgres is currently
requesting succeeds, skip over the trigger check. If the first attempt
fails, then do your trigger check. That way, in the OCF script, the
postmaster can be started, the trigger file set, and connection to the
database looped on until it succeeds as an indication for when the
database is up and available. I think that's cleaner than comparing a
filename from a 'ps' command. Once I've completed the OCF script and
done some testing, I'll forward it to you for you to review and see if
you want to include it.
Thanks,
Doug
On Thu, 2007-03-08 at 15:37 +0000, Simon Riggs wrote:
> On Thu, 2007-03-08 at 10:33 -0500, Doug Knight wrote:
> > Thanks, Simon. I kind of figured that's how pg_standby would work,
> > since its invoked by postgres once per WAL file. What I was thinking I
> > might do in the OCF script is to grab the pg_standby process line from
> > a ps, pull out the "current" WAL file path and filename, then do an
> > existence check for the file. If the file exists, then
> > pg_standby/postgres is probably processing it. If not, then we're
> > probably waiting on it, implying that recovery is complete. Thoughts
> > on this process?
>
> I suppose I might be able to have the option to catch up before it
> stops, on the basis that if it can find the file it was looking for
> without waiting then that can override the trigger.
>
> Which way would you like it to work?
>
From: | "Simon Riggs" <simon(at)2ndquadrant(dot)com> |
---|---|
To: | "Doug Knight" <dknight(at)wsi(dot)com> |
Cc: | <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-03-08 18:34:27 |
Message-ID: | 1173378868.3641.164.camel@silverbirch.site |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
On Thu, 2007-03-08 at 13:29 -0500, Doug Knight wrote:
> I would preserve the existing trigger function as little t "-t", and
> maybe implement a catchup trigger function as big t "-T"? Set it up so
> that if the first attempt to find the WAL file postgres is currently
> requesting succeeds, skip over the trigger check. If the first attempt
> fails, then do your trigger check. That way, in the OCF script, the
> postmaster can be started, the trigger file set, and connection to the
> database looped on until it succeeds as an indication for when the
> database is up and available. I think that's cleaner than comparing a
> filename from a 'ps' command. Once I've completed the OCF script and
> done some testing, I'll forward it to you for you to review and see if
> you want to include it.
I'm happy to do this, unless other objections.
I'll be doing another version before feature freeze.
--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com
From: | Doug Knight <dknight(at)wsi(dot)com> |
---|---|
To: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
Cc: | pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-03-08 19:33:46 |
Message-ID: | 1173382426.4829.50.camel@arc-dknightlx.wsicorp.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
Excellent. Once you're ready, fire it over and I'll test it on our
config.
Doug
On Thu, 2007-03-08 at 18:34 +0000, Simon Riggs wrote:
> On Thu, 2007-03-08 at 13:29 -0500, Doug Knight wrote:
>
> > I would preserve the existing trigger function as little t "-t", and
> > maybe implement a catchup trigger function as big t "-T"? Set it up so
> > that if the first attempt to find the WAL file postgres is currently
> > requesting succeeds, skip over the trigger check. If the first attempt
> > fails, then do your trigger check. That way, in the OCF script, the
> > postmaster can be started, the trigger file set, and connection to the
> > database looped on until it succeeds as an indication for when the
> > database is up and available. I think that's cleaner than comparing a
> > filename from a 'ps' command. Once I've completed the OCF script and
> > done some testing, I'll forward it to you for you to review and see if
> > you want to include it.
>
> I'm happy to do this, unless other objections.
>
> I'll be doing another version before feature freeze.
>
From: | Bruce Momjian <bruce(at)momjian(dot)us> |
---|---|
To: | Simon Riggs <simon(at)2ndquadrant(dot)com> |
Cc: | Doug Knight <dknight(at)wsi(dot)com>, pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: [PATCHES] pg_standby |
Date: | 2007-04-02 22:29:15 |
Message-ID: | 200704022229.l32MTF210810@momjian.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Lists: | pgsql-hackers pgsql-patches |
Simon Riggs wrote:
> On Thu, 2007-03-08 at 13:29 -0500, Doug Knight wrote:
>
> > I would preserve the existing trigger function as little t "-t", and
> > maybe implement a catchup trigger function as big t "-T"? Set it up so
> > that if the first attempt to find the WAL file postgres is currently
> > requesting succeeds, skip over the trigger check. If the first attempt
> > fails, then do your trigger check. That way, in the OCF script, the
> > postmaster can be started, the trigger file set, and connection to the
> > database looped on until it succeeds as an indication for when the
> > database is up and available. I think that's cleaner than comparing a
> > filename from a 'ps' command. Once I've completed the OCF script and
> > done some testing, I'll forward it to you for you to review and see if
> > you want to include it.
>
> I'm happy to do this, unless other objections.
>
> I'll be doing another version before feature freeze.
Should we be getting a patch for this for 8.3?
--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +