Re: numeric and float comparison oddities

From: Kevin Grittner <kgrittn(at)ymail(dot)com>
To: Jeff Davis <pgsql(at)j-davis(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: numeric and float comparison oddities
Date: 2014-08-01 14:50:56
Message-ID: 1406904656.34054.YahooMailNeo@web122305.mail.ne1.yahoo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Jeff Davis <pgsql(at)j-davis(dot)com> wrote:

> I saw some strange results:
>
> postgres=# select '1.1'::numeric = '1.1'::float8;
> ?column?
> ----------
> t
> (1 row)
>
> postgres=# select '1.1'::numeric = '1.1'::float4;
> ?column?
> ----------
> f
> (1 row)

The part I find strange is that the first one evaluates to true,
since numeric can exactly represent 1.1 and float8 cannot.  It also
seems inconsistent with this:

test=# set extra_float_digits = 3;
SET
test=# select '1.1'::float4;
   float4
------------
 1.10000002
(1 row)

test=# select '1.1'::float8;
       float8
---------------------
 1.10000000000000009
(1 row)

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2014-08-01 15:14:35 Re: numeric and float comparison oddities
Previous Message Fujii Masao 2014-08-01 11:35:28 Bug of pg_receivexlog -v