Re: regression failure in CVS HEAD

Lists: pgsql-hackers
From: Neil Conway <neilc(at)samurai(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: regression failure in CVS HEAD
Date: 2003-03-07 20:53:01
Message-ID: 1047070381.13286.2.camel@tokyo
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

About 1 in every 5 runs of the (parallel) regression tests are failing
for me with CVS HEAD: the triggers, inherit, vacuum, sanity_check, and
misc tests fail. I can make the failures occur fairly consistently by
running "make check" over and over again until the problem crops up.

The platform is Linux 2.4, gcc 3.2. I've attached the regression.diffs
file.

Any ideas on what the cause might be?

Cheers,

Neil
--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC

Attachment Content-Type Size
regression.diffs text/x-patch 14.4 KB

From: Doug McNaught <doug(at)mcnaught(dot)org>
To: Neil Conway <neilc(at)samurai(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-07 22:05:16
Message-ID: m365qun883.fsf@varsoon.wireboard.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Neil Conway <neilc(at)samurai(dot)com> writes:

> About 1 in every 5 runs of the (parallel) regression tests are failing
> for me with CVS HEAD: the triggers, inherit, vacuum, sanity_check, and
> misc tests fail. I can make the failures occur fairly consistently by
> running "make check" over and over again until the problem crops up.
>
> The platform is Linux 2.4, gcc 3.2. I've attached the regression.diffs
> file.
>
> Any ideas on what the cause might be?

Hardware?

If it's a software bug, you'd generally expect it to happen each and
every time...

-Doug


From: Rod Taylor <rbt(at)rbt(dot)ca>
To: Doug McNaught <doug(at)mcnaught(dot)org>
Cc: Neil Conway <neilc(at)samurai(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-07 22:28:20
Message-ID: 1047076099.28251.55.camel@jester
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, 2003-03-07 at 17:05, Doug McNaught wrote:
> Neil Conway <neilc(at)samurai(dot)com> writes:
>
> > About 1 in every 5 runs of the (parallel) regression tests are failing
> > for me with CVS HEAD: the triggers, inherit, vacuum, sanity_check, and
> > misc tests fail. I can make the failures occur fairly consistently by
> > running "make check" over and over again until the problem crops up.
> >
> > The platform is Linux 2.4, gcc 3.2. I've attached the regression.diffs
> > file.
> >
> > Any ideas on what the cause might be?
>
> Hardware?
>
> If it's a software bug, you'd generally expect it to happen each and
> every time...

It could be an order dependency in a parallel group.

The sequential runs don't throw a periodic error do they?

--
Rod Taylor <rbt(at)rbt(dot)ca>

PGP Key: http://www.rbt.ca/rbtpub.asc


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Neil Conway <neilc(at)samurai(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-08 05:26:45
Message-ID: 6371.1047101205@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Neil Conway <neilc(at)samurai(dot)com> writes:
> About 1 in every 5 runs of the (parallel) regression tests are failing
> for me with CVS HEAD: the triggers, inherit, vacuum, sanity_check, and
> misc tests fail. I can make the failures occur fairly consistently by
> running "make check" over and over again until the problem crops up.

Yipes. I have not been running the parallel tests (my habit is to run
make installcheck, instead) but there is clearly something busted.
I got a bunch of failures similar to yours in my first attempt with
make check on HPUX --- see attached.

> Any ideas on what the cause might be?

No. Can anyone offer data on when this started?

regards, tom lane

*** ./expected/constraints.out Sat Mar 8 00:16:33 2003
--- ./results/constraints.out Sat Mar 8 00:21:00 2003
***************
*** 11,31 ****
--
CREATE TABLE DEFAULT_TBL (i int DEFAULT 100,
x text DEFAULT 'vadim', f float8 DEFAULT 123.456);
INSERT INTO DEFAULT_TBL VALUES (1, 'thomas', 57.0613);
INSERT INTO DEFAULT_TBL VALUES (1, 'bruce');
INSERT INTO DEFAULT_TBL (i, f) VALUES (2, 987.654);
INSERT INTO DEFAULT_TBL (x) VALUES ('marc');
INSERT INTO DEFAULT_TBL VALUES (3, null, 1.0);
SELECT '' AS five, * FROM DEFAULT_TBL;
! five | i | x | f
! ------+-----+--------+---------
! | 1 | thomas | 57.0613
! | 1 | bruce | 123.456
! | 2 | vadim | 987.654
! | 100 | marc | 123.456
! | 3 | | 1
! (5 rows)
!
CREATE SEQUENCE DEFAULT_SEQ;
CREATE TABLE DEFAULTEXPR_TBL (i1 int DEFAULT 100 + (200-199) * 2,
i2 int DEFAULT nextval('default_seq'));
--- 11,29 ----
--
CREATE TABLE DEFAULT_TBL (i int DEFAULT 100,
x text DEFAULT 'vadim', f float8 DEFAULT 123.456);
+ ERROR: cache lookup of relation 119389 failed
INSERT INTO DEFAULT_TBL VALUES (1, 'thomas', 57.0613);
+ ERROR: Relation "default_tbl" does not exist
INSERT INTO DEFAULT_TBL VALUES (1, 'bruce');
+ ERROR: Relation "default_tbl" does not exist
INSERT INTO DEFAULT_TBL (i, f) VALUES (2, 987.654);
+ ERROR: Relation "default_tbl" does not exist
INSERT INTO DEFAULT_TBL (x) VALUES ('marc');
+ ERROR: Relation "default_tbl" does not exist
INSERT INTO DEFAULT_TBL VALUES (3, null, 1.0);
+ ERROR: Relation "default_tbl" does not exist
SELECT '' AS five, * FROM DEFAULT_TBL;
! ERROR: Relation "default_tbl" does not exist
CREATE SEQUENCE DEFAULT_SEQ;
CREATE TABLE DEFAULTEXPR_TBL (i1 int DEFAULT 100 + (200-199) * 2,
i2 int DEFAULT nextval('default_seq'));

======================================================================

*** ./expected/misc.out Sat Mar 8 00:16:33 2003
--- ./results/misc.out Sat Mar 8 00:21:52 2003
***************
*** 590,596 ****
d_star
date_tbl
default_seq
- default_tbl
defaultexpr_tbl
dept
e_star
--- 590,595 ----
***************
*** 659,665 ****
toyemp
varchar_tbl
xacttest
! (95 rows)

--SELECT name(equipment(hobby_construct(text 'skywalking', text 'mer'))) AS equip_name;
SELECT hobbies_by_name('basketball');
--- 658,664 ----
toyemp
varchar_tbl
xacttest
! (94 rows)

--SELECT name(equipment(hobby_construct(text 'skywalking', text 'mer'))) AS equip_name;
SELECT hobbies_by_name('basketball');

======================================================================


From: Rod Taylor <rbt(at)rbt(dot)ca>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Neil Conway <neilc(at)samurai(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-08 13:36:35
Message-ID: 1047130594.28251.167.camel@jester
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

> Yipes. I have not been running the parallel tests (my habit is to run
> make installcheck, instead) but there is clearly something busted.
> I got a bunch of failures similar to yours in my first attempt with
> make check on HPUX --- see attached.
>
> > Any ideas on what the cause might be?
>
> No. Can anyone offer data on when this started?

I see passes on March 2nd, updated and see passes today.

Did 5 runs of make check on today source.

--
Rod Taylor <rbt(at)rbt(dot)ca>

PGP Key: http://www.rbt.ca/rbtpub.asc


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Neil Conway <neilc(at)samurai(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-08 17:41:03
Message-ID: 7549.1047145263@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

I said:
> Neil Conway <neilc(at)samurai(dot)com> writes:
>> About 1 in every 5 runs of the (parallel) regression tests are failing
>> for me with CVS HEAD: the triggers, inherit, vacuum, sanity_check, and
>> misc tests fail. I can make the failures occur fairly consistently by
>> running "make check" over and over again until the problem crops up.

> Yipes. I have not been running the parallel tests (my habit is to run
> make installcheck, instead) but there is clearly something busted.

I've spent the morning trying to reproduce this, without success. After
a "make maintainer-clean", cvs update, full rebuild cycle, I cannot get
anything funny to happen in "make check" under HPUX, RH Linux 8.0, or
OS X.

I'm a bit hesitant to write it off as a build problem, because
(a) I can't see anything in the recent CVS logs that might cause such,
and (b) it's surprising that it'd bite both you and me. But at this
point I don't see what else to say.

Can you still reproduce the problem after a clean rebuild?

regards, tom lane


From: Joe Conway <mail(at)joeconway(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Neil Conway <neilc(at)samurai(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-08 19:46:34
Message-ID: 3E6A489A.1050307@joeconway.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Tom Lane wrote:
> I've spent the morning trying to reproduce this, without success. After
> a "make maintainer-clean", cvs update, full rebuild cycle, I cannot get
> anything funny to happen in "make check" under HPUX, RH Linux 8.0, or
> OS X.
>
> I'm a bit hesitant to write it off as a build problem, because
> (a) I can't see anything in the recent CVS logs that might cause such,
> and (b) it's surprising that it'd bite both you and me. But at this
> point I don't see what else to say.
>
> Can you still reproduce the problem after a clean rebuild?
>

FWIW, I just updated to cvs tip, did

make clean
./configure ...
make all
make install
make check (repeated 6 times)

The only failure I get is horology due to the
"--enable-integer-datetimes" issue (which I think there is a patch in
the queue for). This is on Red Hat 7.3.

Joe


From: Neil Conway <neilc(at)samurai(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-08 21:56:33
Message-ID: 1047160593.13363.2.camel@tokyo
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Sat, 2003-03-08 at 12:41, Tom Lane wrote:
> Can you still reproduce the problem after a clean rebuild?

No -- I ran "cvs update", "make clean", followed by 10 runs of the
regression tests but I didn't get any similar failures.

I suppose we can just regard it as a build problem, then? Not sure what
the actual culprit was, though...

Cheers,

Neil

--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Neil Conway <neilc(at)samurai(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-09 02:29:46
Message-ID: 16476.1047176986@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Neil Conway <neilc(at)samurai(dot)com> writes:
> I suppose we can just regard it as a build problem, then? Not sure what
> the actual culprit was, though...

I'm mystified too. But unless we see it again, I think we have to write
it off as a build error.

Do you use --enable-depend when configuring? I don't, so I know that
I take some risk of build errors when I do things wrong.

regards, tom lane


From: Neil Conway <neilc(at)samurai(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-09 17:38:02
Message-ID: 1047231482.1206.0.camel@tokyo
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Sat, 2003-03-08 at 21:29, Tom Lane wrote:
> Do you use --enable-depend when configuring? I don't, so I know that
> I take some risk of build errors when I do things wrong.

Yeah, I did -- which I why when I reported it initially I assumed that a
build problem wasn't the cause.

Cheers,

Neil
--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Neil Conway <neilc(at)samurai(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: regression failure in CVS HEAD
Date: 2003-03-09 18:25:50
Message-ID: 20117.1047234350@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Neil Conway <neilc(at)samurai(dot)com> writes:
> On Sat, 2003-03-08 at 21:29, Tom Lane wrote:
>> Do you use --enable-depend when configuring? I don't, so I know that
>> I take some risk of build errors when I do things wrong.

> Yeah, I did -- which I why when I reported it initially I assumed that a
> build problem wasn't the cause.

Hm. One of the reasons I don't use --enable-depend is that I don't
trust it ;-) --- I prefer to do a make clean and rebuild after every
cvs update.

It's disturbing that we both saw similar failures, when there's no
obvious explanation for a build problem in the CVS logs. I have a
sneaking feeling that we haven't seen the last of this issue. But with
no ability to reproduce it, there's not much point in worrying now.

regards, tom lane