Re: 8.0.0beta1: make check fails on solaris8

System Configuration
---------------------
Architecture (example: Intel Pentium) : sun4u sparc
SUNW,Ultra-250

Operating System (example: Linux 2.4.18) : Solaris8 (SunOS
myhost 5.8 Generic_108528-29 )

PostgreSQL version (example: PostgreSQL-8.0): PostgreSQL-8.0.0 (beta1)

Compiler used (example: gcc 2.95.2) : 3.2.3

Please enter a FULL description of your problem:
------------------------------------------------
Errors in regression tests (make check)

Please describe a way to repeat the problem. Please try to provide a
concise reproducible example, if at all possible:
----------------------------------------------------------------------
> ./configure --prefix=/opt/postgresql-800beta1 --enable-thread-safety
> make
> make check

make check gives me two errors, see attached diffs file.

Martin

=?ISO-8859-15?Q?Martin_M=FCnstermann?= <mmuenstermann(at)betrusted(dot)com> writes:
> make check gives me two errors, see attached diffs file.

See my response to Andrea just now:

This seems quite bizarre: ' -INFINiTY ' works but
'infinity' does not? Please try in psql

\set VERBOSITY verbose
SELECT 'infinity'::float4;

and send along the output.

My best guess about it is that strtod() is actively broken on your
platform, and is recognizing the "infinity" input but returning an
incorrect endptr. I seem to recall that we've heard of such bugs
before. Can you check for any updates from Sun that might affect
strtod()?

regards, tom lane

Tom Lane wrote:
> This seems quite bizarre: ' -INFINiTY ' works but
> 'infinity' does not? Please try in psql
>
> \set VERBOSITY verbose
> SELECT 'infinity'::float4;
>
> and send along the output.

It's even a little more bizarre:
When I start a fresh psql session, SELECT 'infinity'::float4 just works.
Then I \i sql/float4.sql, and after that
template1=# \set VERBOSITY verbose
template1=# SELECT 'infinity'::float4;
ERROR: 22P02: invalid input syntax for type real: "infinity"
LOCATION: float4in, float.c:330

The infinity is broken ;-) after SELECT 'NaN'::float4;

BTW: Infinity works again after
SELECT ' -INFINiTY '::float4;

> My best guess about it is that strtod() is actively broken on your
> platform, and is recognizing the "infinity" input but returning an
> incorrect endptr. I seem to recall that we've heard of such bugs
> before. Can you check for any updates from Sun that might affect
> strtod()?

Nothing found yet. But searching the web for "solaris strtod" yields,
that solaris strtod might behave in a special way...

Regards,
Martin

=?ISO-8859-15?Q?Martin_M=FCnstermann?= <mmuenstermann(at)betrusted(dot)com> writes:
> It's even a little more bizarre:
> When I start a fresh psql session, SELECT 'infinity'::float4 just works.
> Then I \i sql/float4.sql, and after that
> template1=# \set VERBOSITY verbose
> template1=# SELECT 'infinity'::float4;
> ERROR: 22P02: invalid input syntax for type real: "infinity"
> LOCATION: float4in, float.c:330

Wow. Well, that confirms my suspicion that endptr is bad --- the line
number shows that the complaint is coming from the
junk-at-end-of-the-string test. But why is it history-dependent?

I wonder if endptr might not be getting set at all in this case.
Could you try adding "endptr = num;" to the code in
src/backend/utils/adt/float.c, that is

/* skip leading whitespace */
while (*num != '\0' && isspace((unsigned char) *num))
num++;

+ endptr = num;
errno = 0;
val = strtod(num, &endptr);

/* did we not see anything that looks like a double? */

at about line 280, and similarly at line 445. Does that make it
any better?

regards, tom lane

>> My best guess about it is that strtod() is actively broken on your
>> platform, and is recognizing the "infinity" input but returning an
>> incorrect endptr. I seem to recall that we've heard of such bugs
>> before. Can you check for any updates from Sun that might affect
>> strtod()?

Hmm, take a look at this:
http://tcl.apache.org/sources/tcl/compat/fixstrtod.c.html

So at least the Tcl guys are aware of an issue. It looks from this code
like they expect Solaris strtod to return an endptr one past the end of
the string. Which would explain the apparently random behavior, because
that would be uninitialized memory --- if it happened to contain a zero
then we'd not register a complaint, but otherwise we would.

So forget my previous proposed patch and try this one:

/* skip leading whitespace */
while (*num != '\0' && isspace((unsigned char) *num))
num++;

errno = 0;
val = strtod(num, &endptr);

+ if (endptr != num && endptr[-1] == 0)
+ endptr--;

/* did we not see anything that looks like a double? */

regards, tom lane

On Wed, Aug 11, 2004 at 09:36:30AM -0400, Tom Lane wrote:
> =?ISO-8859-15?Q?Martin_M=FCnstermann?= <mmuenstermann(at)betrusted(dot)com> writes:
> > It's even a little more bizarre:
> > When I start a fresh psql session, SELECT 'infinity'::float4 just works.
> > Then I \i sql/float4.sql, and after that
> > template1=# \set VERBOSITY verbose
> > template1=# SELECT 'infinity'::float4;
> > ERROR: 22P02: invalid input syntax for type real: "infinity"
> > LOCATION: float4in, float.c:330
>
> Wow. Well, that confirms my suspicion that endptr is bad --- the line
> number shows that the complaint is coming from the
> junk-at-end-of-the-string test. But why is it history-dependent?
>
> I wonder if endptr might not be getting set at all in this case.

I've replied on this topic a couple of times in the "SOLARIS 9
ULTRASPARC BUILD" thread, but perhaps my messages aren't getting
through. On Solaris, when strtod() parses "infinity" or its
equivalent, it sets endptr one character beyond where it should.
This is apparently a longstanding bug that's allegedly been fixed
in Solaris 10:

http://groups.google.com/groups?threadm=4118e611%241_3%40omega.dimensional.com

Here's the example program from the above post to comp.unix.solaris:

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>

int
main(void)
{
double val;
char *endptr;
char num[] = {'i', 'n', 'f', 'i', 'n', 'i', 't', 'y', '\0',
'x', 'y', 'z', '\0'};

errno = 0;
val = strtod(num, &endptr);
printf("val = %f, errno = %d, endptr = %s\n", val, errno, endptr);

return 0;
}

On Solaris 9 this program produces the following output:

val = Inf, errno = 0, endptr = xyz

As the output shows, endptr is set to one character beyond the end
of the "infinity" string. Perhaps the history-dependence in
PostgreSQL is due to endptr being set to point to memory that's
zeroed during the first call to strtod(), but that gets filled with
data before subsequent calls.

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

Michael Fuhr <mike(at)fuhr(dot)org> writes:
> I've replied on this topic a couple of times in the "SOLARIS 9
> ULTRASPARC BUILD" thread, but perhaps my messages aren't getting
> through. On Solaris, when strtod() parses "infinity" or its
> equivalent, it sets endptr one character beyond where it should.
> This is apparently a longstanding bug that's allegedly been fixed
> in Solaris 10:
> http://groups.google.com/groups?threadm=4118e611%241_3%40omega.dimensional.com

Ah, thanks for the link. I had just found out about that bug myself,
but this is more authoritative documentation.

regards, tom lane

On Wed, Aug 11, 2004 at 10:34:24AM -0400, Tom Lane wrote:
> Michael Fuhr <mike(at)fuhr(dot)org> writes:
> > I've replied on this topic a couple of times in the "SOLARIS 9
> > ULTRASPARC BUILD" thread, but perhaps my messages aren't getting
> > through. On Solaris, when strtod() parses "infinity" or its
> > equivalent, it sets endptr one character beyond where it should.
> > This is apparently a longstanding bug that's allegedly been fixed
> > in Solaris 10:
> > http://groups.google.com/groups?threadm=4118e611%241_3%40omega.dimensional.com
>
> Ah, thanks for the link. I had just found out about that bug myself,
> but this is more authoritative documentation.

It's not on groups.google.com yet, but another followup says that
Bug 4751182 is "strtod() with inf or infinity input sets endptr
incorrectly".

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

Tom Lane wrote:
> So forget my previous proposed patch and try this one:
>
> /* skip leading whitespace */
> while (*num != '\0' && isspace((unsigned char) *num))
> num++;
>
> errno = 0;
> val = strtod(num, &endptr);
>
> + if (endptr != num && endptr[-1] == 0)
> + endptr--;
>
> /* did we not see anything that looks like a double? */

Yeah, with this patch the tests just pass on solaris8.

--- float.c.orig Wed Aug 4 23:34:02 2004
+++ float.c Wed Aug 11 16:37:00 2004
@@ -282,6 +282,9 @@
errno = 0;
val = strtod(num, &endptr);

+ if (endptr != num && endptr[-1] == 0)
+ endptr--;
+
/* did we not see anything that looks like a double? */
if (endptr == num || errno != 0)
{
@@ -446,6 +449,10 @@

errno = 0;
val = strtod(num, &endptr);
+
+ if (endptr != num && endptr[-1] == 0)
+ endptr--;
+

/* did we not see anything that looks like a double? */
if (endptr == num || errno != 0)

Regards,
Martin Muenstermann

=?ISO-8859-15?Q?Martin_M=FCnstermann?= <mmuenstermann(at)betrusted(dot)com> writes:
> Yeah, with this patch the tests just pass on solaris8.

Great, thanks for the confirmation.

regards, tom lane

I've applied the attached patch to try to fix this problem with minimal
run-time cost. Since I don't have a Solaris machine I can't positively
confirm it works --- would one of you please test?

regards, tom lane

*** src/backend/utils/adt/float.c.orig Wed Aug 4 17:34:02 2004
--- src/backend/utils/adt/float.c Wed Aug 11 13:16:58 2004
***************
*** 317,322 ****
--- 317,334 ----
errmsg("invalid input syntax for type real: \"%s\"",
orig_num)));
}
+ #ifdef HAVE_BUGGY_SOLARIS_STRTOD
+ else
+ {
+ /*
+ * Many versions of Solaris have a bug wherein strtod sets endptr
+ * to point one byte beyond the end of the string when given
+ * "inf" or "infinity".
+ */
+ if (endptr != num && endptr[-1] == '\0')
+ endptr--;
+ }
+ #endif /* HAVE_BUGGY_SOLARIS_STRTOD */

/* skip trailing whitespace */
while (*endptr != '\0' && isspace((unsigned char) *endptr))
***************
*** 482,487 ****
--- 494,511 ----
errmsg("invalid input syntax for type double precision: \"%s\"",
orig_num)));
}
+ #ifdef HAVE_BUGGY_SOLARIS_STRTOD
+ else
+ {
+ /*
+ * Many versions of Solaris have a bug wherein strtod sets endptr
+ * to point one byte beyond the end of the string when given
+ * "inf" or "infinity".
+ */
+ if (endptr != num && endptr[-1] == '\0')
+ endptr--;
+ }
+ #endif /* HAVE_BUGGY_SOLARIS_STRTOD */

/* skip trailing whitespace */
while (*endptr != '\0' && isspace((unsigned char) *endptr))
*** src/include/port/solaris.h.orig Sun Mar 14 22:29:22 2004
--- src/include/port/solaris.h Wed Aug 11 13:16:50 2004
***************
*** 35,37 ****
--- 35,44 ----
#define BYTE_ORDER LITTLE_ENDIAN
#endif
#endif
+
+ /*
+ * Many versions of Solaris have broken strtod() --- see bug #4751182.
+ * For the moment we just assume they all do; it's probably not worth
+ * the trouble to add a configure test for this.
+ */
+ #define HAVE_BUGGY_SOLARIS_STRTOD

On Wed, Aug 11, 2004 at 01:22:46PM -0400, Tom Lane wrote:
> I've applied the attached patch to try to fix this problem with minimal
> run-time cost. Since I don't have a Solaris machine I can't positively
> confirm it works --- would one of you please test?

======================
All 96 tests passed.
======================

I've run numerous queries with variants of 'Infinity' and '-Infinity'
and I haven't been able to break it. Thanks for taking care of it.

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

Michael Fuhr <mike(at)fuhr(dot)org> writes:
> I've run numerous queries with variants of 'Infinity' and '-Infinity'
> and I haven't been able to break it. Thanks for taking care of it.

No problem. Appreciate the followup.

regards, tom lane