Re: WAL CPU overhead/optimization (was Master-slave visibility order)

On Fri, Aug 30, 2013 at 03:22:37AM +0300, Ants Aasma wrote:
> On Fri, Aug 30, 2013 at 3:02 AM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> > I am not sure "hot cache large buffer performance" is really the
> > interesting case. Most of the XLogInsert()s are pretty small in the
> > common workloads. I vaguely recall trying 8 and getting worse
> > performance on many workloads, but that might have been a problem of my
> > implementation.
>
> Slice-by-8 doesn't have any overhead for small buffers besides the
> lookup tables, so it most likely the cache misses that were the issue.
> Murmur3, CityHash and SpookyHash don't have any lookup tables and are
> excellent with small keys. Especially CityHash, 64 byte hash is quoted
> at 9ns.
>
> > The reason I'd like to go for a faster CRC32 implementation as a first
> > step is that it's easy. Easy to verify, easy to analyze, easy to
> > backout. I personally don't have enough interest/time in the 9.4 cycle
> > to purse conversion to a different algorithm (I find the idea of using
> > different ones on 32/64bit pretty bad), but I obviously won't stop
> > somebody else ;)
>
> I might give it a shot later this cycle as I have familiarized my self
> with the problem domain anyway. I understand the appeal of staying
> with what we have, but this would cap the speedup at 4x and has large
> caveats with the extra lookup tables. A 28x speedup might be worth the
> extra effort.
>
> Regards,
> Ants Aasma
>

You may want to also check out xxhash with a BSD License and very fast
32-bit performance as well:

http://fastcompression.blogspot.com/2012/04/selecting-checksum-algorithm.html
http://code.google.com/p/xxhash/

FWIW I agree that a much faster function would be better for CPU overhead.

Regards,
Ken