Re: Floating point error

On 02/24/2013 06:58 PM, Tom Duffey wrote:
>
> On Feb 24, 2013, at 8:44 PM, Adrian Klaver <adrian(dot)klaver(at)gmail(dot)com> wrote:
>
>> On 02/24/2013 06:13 PM, Tom Duffey wrote:
>>> Hi Everyone,
>>>
>>> Riddle me this. I have a database column of type "real" that gets mapped to a Java field of type double via JDBC. We have two databases, test and production, and the test database is periodically blown away and reloaded from a copy of production. We recently noticed that some values do not match when viewed within our application on test vs. production. More specifically:
>>>
>>> - Selecting values from both test and production DBs using psql shows "10.3885" as the value
>>> - The Java app on production shows "10.3884573" while the test app shows "10.3885"
>>>
>>> I have a hunch that when the value was originally inserted into the production DB it probably contained more than the 6 digits supported by the real data type. It may have even been exactly the "10.3884573" value we see when retrieving via JDBC on production. What I don't understand is why when the value gets mapped back to Java via JDBC those extra digits are coming back. Can anyone explain this or do you think I'm on the wrong track? I stepped through code and it sure seems like the extra information is coming back from the JDBC driver.
>>
>> Are the production and test apps running on the same platform i.e. OS, bitness, etc.
>
> Yes, the production and test apps are running on the same platform. The Java apps themselves are physically on the same Linux server. The production and test databases reside within the same instance of PostgreSQL.
>
> Also, I should have mentioned up front that I am well aware of the pitfalls of using floating point values and also the fact that PostgreSQL's "real" data type supports 6 digits of precision. What I do not understand is why my JDBC driver is returning more information than what I receive in psql or if I operate on a copy of the database. This leads me to believe that more information was available at insertion time and is somehow being made available to my application even though the data type should only store 6 digits. Let me see if I can write a quick little test case.
>

Well I guess you could look in the dump file and see what is recorded there.

> Tom
>

--
Adrian Klaver
adrian(dot)klaver(at)gmail(dot)com