Re: Enabling Checksums

On 10 April 2013 11:15, Ants Aasma <ants(at)cybertec(dot)at> wrote:

> > * We might even be able to calculate CRC32 checksum for normal WAL
> records,
> > and use Ants' checksum for full page writes (only). So checking WAL
> checksum
> > would then be to confirm header passes CRC32 and then re-check the Ants
> > checksum of each backup block.
>
> If we ensure that the checksum on the page is correct when we do a
> full page write then we could only include the checksum field in the
> WAL CRC. When reading WAL we would first check that the CRC is correct
> and then verify the the page checksum.

OK, so we have a single combined "calculate a checksum for a block"
function. That uses Jeff's zeroing trick and Ants' bulk-oriented
performance optimization.

For buffer checksums we simply calculate for the block.

For WAL full page writes, we first set the checksums for all defined
buffers, then calculate the checksum of remaining data plus the pd_checksum
field from each block using the normal WAL CRC32.

Seems good to me. One set of fast code. And it avoids the weirdness that
the checksum stored on the full page is actually wrong.

It also means that the WAL checksum calculation includes the hole, yet we
do not include the data for the hole. So we have to do an extra copy when
restoring the backuo block.

Comments?

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services