Re: Checkpoint Location Format

Hi,

I'm writing an automated file-system level backup application for use
with WAL archiving, that will issue the pg_start_backup call, tar and
gzip the cluster data directory, issue the pg_stop_backup call, and
remove all previous un-needed WAL files from the archive.

I need to write a regular expression that will search for the WAL
filename and checkpoint location from the backup_label file, and just
want to clarify that the checkpoint location will always be of the
format: X/XXXXXX - where X is one of 0-9, A-F?

And then the WAL .backup file that is generated in the archive, has a
filename of the form:

<WAL_FILE>.00XXXXXX.backup

where <WAL_FILE> is the name of the "STARTING WAL LOCATION" directive in
the backup_label file, and XXXXXX is the last 6 digits of the checkpoint
(after the / )?

Thanks,

Andy.

On Tue, Nov 07, 2006 at 01:12:53PM +0000, Andy Shellam (Mailing Lists) wrote:
> Hi,
>
> I'm writing an automated file-system level backup application for use
> with WAL archiving, that will issue the pg_start_backup call, tar and
> gzip the cluster data directory, issue the pg_stop_backup call, and
> remove all previous un-needed WAL files from the archive.
>
> I need to write a regular expression that will search for the WAL
> filename and checkpoint location from the backup_label file, and just
> want to clarify that the checkpoint location will always be of the
> format: X/XXXXXX - where X is one of 0-9, A-F?
>
> And then the WAL .backup file that is generated in the archive, has a
> filename of the form:
>
> <WAL_FILE>.00XXXXXX.backup
>
> where <WAL_FILE> is the name of the "STARTING WAL LOCATION" directive in
> the backup_label file, and XXXXXX is the last 6 digits of the checkpoint
> (after the / )?

I don't know the answer, but since no one's replied I suggest looking in
the code. Looking at the source of pg_start_backup would probably be a
good start, though I'm guessing the real answer is somewhere in the
backend.
--
Jim Nasby jim(at)nasby(dot)net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Hi Jim,

Tom did answer actually! (Although it was first thing in the morning
and he hadn't had any caffeine so he may have forgot to copy the list in
;) )
I forget what he said the format was now, and I haven't got his e-mail,
but I've just done a regex to match one or more characters before and
after the "/", which works.

Andy.

Jim C. Nasby wrote:
> On Tue, Nov 07, 2006 at 01:12:53PM +0000, Andy Shellam (Mailing Lists) wrote:
>
>> Hi,
>>
>> I'm writing an automated file-system level backup application for use
>> with WAL archiving, that will issue the pg_start_backup call, tar and
>> gzip the cluster data directory, issue the pg_stop_backup call, and
>> remove all previous un-needed WAL files from the archive.
>>
>> I need to write a regular expression that will search for the WAL
>> filename and checkpoint location from the backup_label file, and just
>> want to clarify that the checkpoint location will always be of the
>> format: X/XXXXXX - where X is one of 0-9, A-F?
>>
>> And then the WAL .backup file that is generated in the archive, has a
>> filename of the form:
>>
>> <WAL_FILE>.00XXXXXX.backup
>>
>> where <WAL_FILE> is the name of the "STARTING WAL LOCATION" directive in
>> the backup_label file, and XXXXXX is the last 6 digits of the checkpoint
>> (after the / )?
>>
>
> I don't know the answer, but since no one's replied I suggest looking in
> the code. Looking at the source of pg_start_backup would probably be a
> good start, though I'm guessing the real answer is somewhere in the
> backend.
>

On Tue, 2006-11-07 at 13:12 +0000, Andy Shellam (Mailing Lists) wrote:

> I'm writing an automated file-system level backup application for use
> with WAL archiving, that will issue the pg_start_backup call, tar and
> gzip the cluster data directory, issue the pg_stop_backup call, and
> remove all previous un-needed WAL files from the archive.

Is this any help?

http://archives.postgresql.org/pgsql-patches/2006-05/msg00229.php

If so, I'll see about updating it so it can get backpatched to 8.0 and
8.1 also.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com

Yup that sounds useful - would this be another column added to the
pg_stop_backup called "wal_filename" or similar?
My script uses this name to find the date/time of the current ".backup"
file and remove any files earlier than it (except obviously the one
listed in the START WAL LOCATION).
I've yet to run a restore test, but the backup and automatic archive
clearout is working nicely.

I'm not too worried about the SQL function to find the current WAL file,
although I can see this could be useful too.

Thanks,

Andy.

Simon Riggs wrote:
> On Tue, 2006-11-07 at 13:12 +0000, Andy Shellam (Mailing Lists) wrote:
>
>
>> I'm writing an automated file-system level backup application for use
>> with WAL archiving, that will issue the pg_start_backup call, tar and
>> gzip the cluster data directory, issue the pg_stop_backup call, and
>> remove all previous un-needed WAL files from the archive.
>>
>
> Is this any help?
>
> http://archives.postgresql.org/pgsql-patches/2006-05/msg00229.php
>
> If so, I'll see about updating it so it can get backpatched to 8.0 and
> 8.1 also.
>
>

--
Andy Shellam
NetServe Support Team

the Mail Network
"an alternative in a standardised world"

p: +44 (0) 121 288 0832/0839
m: +44 (0) 7818 000834