Re: CRC algorithm (was Re: [REVIEW] Re: Compression of full-page-writes)

Em 12/09/2014 17:23, "Andres Freund" <andres(at)2ndquadrant(dot)com> escreveu:
>
> On 2014-09-12 23:03:00 +0300, Heikki Linnakangas wrote:
> > On 09/12/2014 10:54 PM, Abhijit Menon-Sen wrote:
> > >At 2014-09-12 22:38:01 +0300, hlinnakangas(at)vmware(dot)com wrote:
> > >>
> > >>We probably should consider switching to a faster CRC algorithm again,
> > >>regardless of what we do with compression.
> > >
> > >As it happens, I'm already working on resurrecting a patch that Andres
> > >posted in 2010 to switch to zlib's faster CRC implementation.
> >
> > As it happens, I also wrote an implementation of Slice-by-4 the other
day
> > :-). Haven't gotten around to post it, but here it is.
> >
> > What algorithm does zlib use for CRC calculation?
>
> Also slice-by-4, with a manually unrolled loop doing 32bytes at once,
using
> individual slice-by-4's. IIRC I tried and removing that slowed things
> down overall. What it also did was move crc to a function. I'm not sure
> why I did it that way, but it really might be beneficial - if you look
> at profiles today there's sometimes icache/decoding stalls...
>
> Hm. Let me look:
>
http://archives.postgresql.org/message-id/201005202227.49990.andres%40anarazel.de
>
> Ick, there's quite some debugging leftovers ;)
>
> I think it might be a good idea to also switch the polynom at the same
> time. I really really think we should, when the hardware supports, use
> the polynom that's available in SSE4.2. It has similar properties, can
> implemented in software just the same...
>
> Greetings,
>
> Andres Freund
>
> --
> Andres Freund http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

This Google library is worth a look https://code.google.com/p/crcutil/ as
it has some extremely optimized versions.