Re: [REVIEW] Re: Compression of full-page-writes

On 09/02/2014 09:52 AM, Fujii Masao wrote:
> [RESULT]
> Throughput in the benchmark.
>
> Multiple Single
> off 2162.6 2164.5
> on 891.8 895.6
> pglz 1037.2 1042.3
> lz4 1084.7 1091.8
> snappy 1058.4 1073.3

Most of the CPU overhead of writing full pages is because of CRC
calculation. Compression helps because then you have less data to CRC.

It's worth noting that there are faster CRC implementations out there
than what we use. The Slicing-by-4 algorithm was discussed years ago,
but was not deemed worth it back then IIRC because we typically
calculate CRC over very small chunks of data, and the benefit of
Slicing-by-4 and many other algorithms only show up when you work on
larger chunks. But a full-page image is probably large enough to benefit.

What I'm trying to say is that this should be compared with the idea of
just switching the CRC implementation. That would make the 'on' case
faster, and and the benefit of compression smaller. I wouldn't be
surprised if it made the 'on' case faster than compressed cases.

I don't mean that we should abandon this patch - compression makes the
WAL smaller which has all kinds of other benefits, even if it makes the
raw TPS throughput of the system worse. But I'm just saying that these
TPS comparisons should be taken with a grain of salt. We probably should
consider switching to a faster CRC algorithm again, regardless of what
we do with compression.

- Heikki