Allow pg_archivecleanup to ignore extensions

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Allow pg_archivecleanup to ignore extensions
Date: 2011-02-08 07:57:42
Message-ID: 4D50F776.6080701@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

One bit of feedback I keep getting from people who archive their WAL
files is that the fairly new pg_archivecleanup utility doesn't handle
the case where those archives are compressed. As the sort of users who
are concerned about compression are also often ones with giant archives
they struggle to cleanup, they would certainly appreciate having a
bundled utility to take care of that.

The attached patch provides an additional option to the utility to
provide this capability. It just strips a provided extension off any
matching file it considers before running the test for whether it should
be deleted or not. It includes updates to the usage message and some
docs about how this might be used. Code by Jaime Casanova and myself.

Here's an example of it working:

$ psql -c "show archive_command"
archive_command
----------------------------------------------------
cp -i %p archive/%f < /dev/null && gzip archive/%f
[Yes, I know that can be written more cleanly. I call external scripts
with more serious error handling than you can put into a single command
line for this sort of thing in production.]

$ psql -c "select pg_start_backup('test',true)"
$ psql -c "select pg_stop_backup()"
$ psql -c "checkpoint"
$ psql -c "select pg_switch_xlog()"

$ cd $PGDATA/archive
$ ls
000000010000000000000025.gz
000000010000000000000026.gz
000000010000000000000027.gz
000000010000000000000028.00000020.backup.gz
000000010000000000000028.gz
000000010000000000000029.gz

$ pg_archivecleanup -d -x .gz `pwd`
000000010000000000000028.00000020.backup
pg_archivecleanup: keep WAL file
"/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000028"
and later
pg_archivecleanup: removing file
"/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000025.gz"
pg_archivecleanup: removing file
"/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000027.gz"
pg_archivecleanup: removing file
"/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000026.gz"
$ ls
000000010000000000000028.00000020.backup.gz
000000010000000000000028.gz
000000010000000000000029.gz

We recenty got some on-list griping that pg_standby doesn't handle
archive files that are compressed, either. Given how the job I'm
working on this week is going, I'll probably have to add that feature
next. That's actually an easier source code hack than this one, because
of how the pg_standby code modularizes the concept of a restore command.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us

Attachment Content-Type Size
archivecleanup_extensions-v2.patch text/x-diff 5.2 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kevin Grittner 2011-02-08 08:43:06 Re: SSI patch version 14
Previous Message Nick Rudnick 2011-02-08 07:23:15 Re: [pgsql-general 2011-1-21:] Are there any projects interested in object functionality? (+ rule bases)