Re: Enabling Checksums

On Sun, 2013-03-03 at 18:05 -0500, Greg Smith wrote:
> = Test 1 - find worst-case overhead for the checksum calculation on write =
>
> This can hit 25% of runtime when you isolate it out. I'm not sure if
> how I'm running this multiple times makes sense yet. This one is so
> much slower on my Mac that I can't barely see a change at all.
>
> = Test 2 - worst-case overhead for calculating checksum while reading data =
>
> Jeff saw an 18% slowdown, I get 24 to 32%. This one bothers me because
> the hit is going to happen during the very common situation where data
> is shuffling a lot between a larger OS cache and shared_buffers taking a
> relatively small fraction. If that issue were cracked, such that
> shared_buffers could be >50% of RAM, I think the typical real-world
> impact of this would be easier to take.

I believe that test 1 and test 2 can be improved a little, if there is a
need. Right now we copy the page and then calculate the checksum on the
copy. If we instead calculate as we're copying, I believe it will make
it significantly faster.

I decided against doing that, because it decreased the readability, and
we can always do that later as an optimization. That should mitigate the
case you have in mind, which is a very legitimate concern. I'll wait for
someone to ask for it, though.

> = Test 3 - worst-case WAL overhead =
>
> This is the really nasty one. The 10,000,000 rows touched by the SELECT
> statement here create no WAL in a non-checksum environment. When
> checksums are on, 368,513,656 bytes of WAL are written, so about 37
> bytes per row.

Yeah, nothing we can do about this.

> Right now the whole hint bit mechanism and its overhead are treated as
> an internal detail that isn't in the regular documentation. I think
> committing this sort of checksum patch will require exposing some of the
> implementation to the user in the documentation, so people can
> understand what the trouble cases are--either in advance or when trying
> to puzzle out why they're hitting one of them.

Any particular sections that you think would be good to update?

Thank you for the test results.

Regards,
Jeff Davis