Re: [PATCH] Negative Transition Aggregate Functions (WIP)

On Jan9, 2014, at 17:15 , Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> writes:
>> Reading over this, I realised that there is a problem with NaN
>> handling --- once the state becomes NaN, it can never recover. So the
>> results using the inverse transition function don't match HEAD in
>> cases like this:
>
> Ouch! That takes out numeric, float4, and float8 in one fell swoop.

For numeric, it seems that this could be overcome by having the state
be a pair (s numeric, n numeric). s would track the sum of non-NaNs
summands and n would track the number of NaN summands. The final
function would return NaN if n > 0 and s otherwise. The pair could
be represented as a value of type numeric[] to avoid having to invent
a new type for this.

For float 4 and float8, wasn't the consensus that the potential
lossy-ness of addition makes this impossible anyway, even without the
NaN issue? But...

> Given the relative infrequency of NaNs in most data, it seems like
> it might still be possible to get a speedup if we could use inverse
> transitions until we hit a NaN, then do it the hard way until the
> NaN is outside the window, then go back to inverse transitions.
> I'm not sure though if this is at all practical from an implementation
> standpoint. We certainly don't want the core code knowing about
> anything as datatype-specific as a NaN, but maybe the inverse transition
> function could have an API that allows reporting "I can't do it here,
> fall back to the hard way".

that sounds like it might be possible to make things work for float4
and float8 afterall, if we can determine whether a particular addition
was lossy or not.

best regards,
Florian Pflug