Re: About numeric division again

"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> One of the items on the commit-fest list is my patch from last year
> to rewrite the numeric division operator using "schoolbook division":
> http://archives.postgresql.org/pgsql-patches/2007-06/msg00173.php
>
> The code that's currently in there sometimes has to propagate rounding
> to the left, meaning that you can never be certain whether all of the
> digits you have so far are good, and that means that it can sometimes
> generate an incorrect truncated output. This leads to the bugs cited
> in the above message.

The case I looked into could be traced specifically to the fact that it
calculates an intermediate value which is the reciprocal of the divisor and
multiplies by that. The case where the digits were inaccurate was a case where
there reciprocal wasn't representable and multiplying by the reciprocal didn't
produce the same value as dividing.

I assume you're right about there being bigger problems but I don't follow how
the division is actually being done in enough detail to judge that for my
self.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's 24x7 Postgres support!