Re: Checksums by default?

On Sat, Jan 21, 2017 at 10:16 PM, Michael Banck
<michael(dot)banck(at)credativ(dot)de> wrote:
> On Sat, Jan 21, 2017 at 09:02:25PM +0200, Ants Aasma wrote:
>> On Sat, Jan 21, 2017 at 6:41 PM, Andreas Karlsson <andreas(at)proxel(dot)se> wrote:
>> > It might be worth looking into using the CRC CPU instruction to reduce this
>> > overhead, like we do for the WAL checksums. Since that is a different
>> > algorithm it would be a compatibility break and we would need to support the
>> > old algorithm for upgraded clusters..
>>
>> We looked at that when picking the algorithm. At that point it seemed
>> that CRC CPU instructions were not universal enough to rely on them.
>> The algorithm we ended up on was designed to be fast on SIMD hardware.
>> Unfortunately on x86-64 that required SSE4.1 integer instructions, so
>> with default compiles there is a lot of performance left on table. A
>> low hanging fruit would be to do CPU detection like the CRC case and
>> enable a SSE4.1 optimized variant when those instructions are
>> available. IIRC it was actually a lot faster than the naive hardware
>> CRC that is used for WAL and about on par with interleaved CRC.
>
> I am afraid that won't fly with most end-user packages, cause
> distributions can't just built packages on their newest machine and then
> users get SIGILL or whatever cause their 2014 server doesn't have that
> instruction, or would they still work?
>
> So you would have to do runtime detection of the CPU features, and use
> the appropriate code if they are available. Which also makes regression
> testing harder, as not all codepaths would get exercised all the time.

Runtime detection is exactly what I had in mind. The code path would
also be the same as at least the two most important compilers only
need a compilation flag change. And the required instruction was
introduced in 2007 so I think anybody who is concerned about
performance is covered.

Regards,
Ants Aasma