Re: pg_filedump 9.3: checksums (and a few other fixes)

On 2013-06-27 09:51:07 -0400, Tom Lane wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> > On 2013-06-26 21:18:49 -0700, Peter Geoghegan wrote:
> >> Heroku are interested in online verification of basebackups (i.e.
> >> using checksums to verify the integrity of heap files as they are
> >> backed up, with a view to relying less and less on logical backups).
>
> > Why not do this from a function/background worker in the backend where
> > you can go via the buffer manager to avoid torn pages et al. If you use
> > a buffer strategy the cache poisoning et al should be controlleable.
>
> That would require having a functioning postmaster environment, which
> seems like it would make such a tool much less flexible.

I personally wouldn't trust online backups that aren't proven to be
replayable into a runnable state very much. I have seen too many cases
where that failed.

Maybe the trick is to add a recovery.conf option to make postgres replay
to the first restartpoint and then shutdown. At that point you can be
sure there aren't any torn pages anymore (bugs aside).
In fact that sounds like a rather useful pg_basebackup extension...

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services