Re: build farm machine using <make -j 8> mixed results

I change the build-farm.conf file to have the following make line:

make => 'make -j 8', # or gmake if required. can include path if necessary.

2 pass, 4 fail. Is this a build configuration you want to pursue? I can either create a new machine, or change one of my existing machines. Makes no difference to me.

roberts-imac:build-farm-4.7-j robert$ time ./run_branches.pl --run-all --test
Sat Sep 1 09:26:49 2012: buildfarm run for CHANGEME:HEAD starting
[09:26:49] checking out source ...
[09:27:59] checking if build run needed ...
[09:27:59] copying source to pgsql.54279 ...
[09:28:03] running configure ...
[09:28:25] running make ...
[09:28:45] running make check ...
Branch: HEAD
Stage Check failed with status 2
Sat Sep 1 09:29:09 2012: buildfarm run for CHANGEME:REL9_2_STABLE starting
[09:29:09] checking out source ...
[09:29:57] checking if build run needed ...
[09:29:57] copying source to pgsql.70926 ...
[09:29:58] running configure ...
[09:30:18] running make ...
[09:30:45] running make check ...
[09:31:09] running make contrib ...
[09:31:12] running make install ...
[09:31:15] running make contrib install ...
[09:31:16] setting up db cluster (C)...
[09:31:19] starting db (C)...
[09:31:20] running make installcheck (C)...
[09:31:39] restarting db (C)...
[09:31:51] running make isolation check ...
[09:32:00] restarting db (C)...
[09:32:12] running make PL installcheck (C)...
Branch: REL9_2_STABLE
Stage PLCheck-C failed with status 2
Sat Sep 1 09:32:29 2012: buildfarm run for CHANGEME:REL9_1_STABLE starting
[09:32:29] checking out source ...
[09:33:08] checking if build run needed ...
[09:33:08] copying source to pgsql.94883 ...
[09:33:10] running configure ...
[09:33:30] running make ...
[09:33:54] running make check ...
[09:34:18] running make contrib ...
[09:34:21] running make install ...
[09:34:23] running make contrib install ...
[09:34:24] setting up db cluster (C)...
[09:34:26] starting db (C)...
[09:34:27] running make installcheck (C)...
[09:34:47] restarting db (C)...
[09:34:59] running make isolation check ...
[09:35:16] restarting db (C)...
[09:35:28] running make PL installcheck (C)...
Branch: REL9_1_STABLE
Stage PLCheck-C failed with status 2
Sat Sep 1 09:35:46 2012: buildfarm run for CHANGEME:REL9_0_STABLE starting
[09:35:46] checking out source ...
[09:36:08] checking if build run needed ...
[09:36:08] copying source to pgsql.18851 ...
[09:36:10] running configure ...
[09:36:28] running make ...
[09:37:00] running make check ...
[09:37:23] running make contrib ...
[09:37:27] running make install ...
[09:37:30] running make contrib install ...
[09:37:32] setting up db cluster (C)...
[09:37:34] starting db (C)...
[09:37:35] running make installcheck (C)...
[09:37:52] restarting db (C)...
[09:38:04] running make PL installcheck (C)...
[09:38:06] restarting db (C)...
[09:38:18] running make contrib installcheck (C)...
[09:38:30] stopping db (C)...
[09:38:31] running make ecpg check ...
[09:38:43] OK
Branch: REL9_0_STABLE
All stages succeeded
Sat Sep 1 09:38:43 2012: buildfarm run for CHANGEME:REL8_4_STABLE starting
[09:38:43] checking out source ...
[09:38:57] checking if build run needed ...
[09:38:57] copying source to pgsql.45071 ...
[09:38:59] running configure ...
[09:39:19] running make ...
[09:39:46] running make check ...
[09:40:14] running make contrib ...
[09:40:17] running make install ...
[09:40:23] running make contrib install ...
[09:40:25] setting up db cluster (C)...
[09:40:30] starting db (C)...
[09:40:31] running make installcheck (C)...
[09:40:47] restarting db (C)...
[09:40:59] running make PL installcheck (C)...
[09:41:01] restarting db (C)...
[09:41:13] running make contrib installcheck (C)...
[09:41:25] stopping db (C)...
[09:41:26] running make ecpg check ...
[09:41:44] OK
Branch: REL8_4_STABLE
All stages succeeded
Sat Sep 1 09:41:44 2012: buildfarm run for CHANGEME:REL8_3_STABLE starting
[09:41:44] checking out source ...
[09:42:39] checking if build run needed ...
[09:42:39] copying source to pgsql.80222 ...
[09:42:42] running configure ...
[09:43:03] running make ...
[09:43:38] running make check ...
[09:44:04] running make contrib ...
[09:44:09] running make install ...
[09:44:14] running make contrib install ...
[09:44:17] setting up db cluster (C)...
[09:44:20] starting db (C)...
[09:44:21] running make installcheck (C)...
[09:44:39] restarting db (C)...
[09:44:51] running make PL installcheck (C)...
[09:44:52] restarting db (C)...
[09:45:05] running make contrib installcheck (C)...
Branch: REL8_3_STABLE
Stage ContribCheck-C failed with status 2

Excerpts from Robert Creager's message of sáb sep 01 12:12:51 -0400 2012:
>
> I change the build-farm.conf file to have the following make line:
>
> make => 'make -j 8', # or gmake if required. can include path if necessary.
>
> 2 pass, 4 fail. Is this a build configuration you want to pursue?

Sure, why not? Parallel building is going to become more common, so
it's a good idea to investigate the failures. I would have it build
only HEAD though, because we're not likely to backpatch these fixes.

> I can either create a new machine, or change one of my existing machines. Makes no difference to me.

If we want to have it run only HEAD I would say you should create a new
animal.

Perhaps you should wait until after 9.2 has been released, though, to
avoid distracting people :-)

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 9/1/12 12:12 PM, Robert Creager wrote:
>
> I change the build-farm.conf file to have the following make line:
>
> make => 'make -j 8', # or gmake if required. can include path if
> necessary.
>
> 2 pass, 4 fail. Is this a build configuration you want to pursue?

Sure that would be useful, but it's pretty clear that the check stages
don't work in parallel. It think it's because the ports conflict, but
there could be any number of other problems.

That said, it would be useful, in my mind, to support parallel checks.
But unless someone is going to put in the work first, you should
restrict your parallel runs to the build and install phases.

Excerpts from Peter Eisentraut's message of mar sep 04 18:49:46 -0300 2012:
> On 9/1/12 12:12 PM, Robert Creager wrote:
> >
> > I change the build-farm.conf file to have the following make line:
> >
> > make => 'make -j 8', # or gmake if required. can include path if
> > necessary.
> >
> > 2 pass, 4 fail. Is this a build configuration you want to pursue?
>
> Sure that would be useful, but it's pretty clear that the check stages
> don't work in parallel. It think it's because the ports conflict, but
> there could be any number of other problems.

Is that really the problem? As far as I know, buildfarm doesn't use
anything like installcheck-world or similar targets; each check target
is run separately, serially, by the BF script.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On 09/04/2012 05:49 PM, Peter Eisentraut wrote:
> On 9/1/12 12:12 PM, Robert Creager wrote:
>> I change the build-farm.conf file to have the following make line:
>>
>> make => 'make -j 8', # or gmake if required. can include path if
>> necessary.
>>
>> 2 pass, 4 fail. Is this a build configuration you want to pursue?
> Sure that would be useful, but it's pretty clear that the check stages
> don't work in parallel. It think it's because the ports conflict, but
> there could be any number of other problems.
>
> That said, it would be useful, in my mind, to support parallel checks.
> But unless someone is going to put in the work first, you should
> restrict your parallel runs to the build and install phases.
>
>

The buildfarm code doesn't contain a facility to use a different make
incantation for each step. It's pretty much an all or nothing deal. Of
course, you can hack run_build.pl to make it do that, but I highly
discourage that. For one thing, it makes upgrading that much more
difficult. All the tweaking is supposed to be done vie the config file.
I guess I could add a setting that allowed for per step make flags.

Frankly, I have had enough failures of parallel make that I think doing
this would generate a significant number of non-repeatable failures (I
had one just the other day that took three invocations of make to get
right). So I'm not sure doing this would advance us much, although I'm
open to persuasion.

cheers

andrew

On Sep 4, 2012 6:06 PM, "Andrew Dunstan" <andrew(at)dunslane(dot)net> wrote:
>
>
> Frankly, I have had enough failures of parallel make that I think doing
this would generate a significant number of non-repeatable failures (I had
one just the other day that took three invocations of make to get right).
So I'm not sure doing this would advance us much, although I'm open to
persuasion.

Seeing as most PostgreSQL bugs appear with concurrency, I think we should
leave our default config with 1 for max connections.

;-)

Parallel make failures are bugs in the dependencies as described in our
make files.

For the build phase, I don't recall parallel problems and as a habit I
usually use parallel makes. I would like that to be supported and I think
I've seen fixes applied when we had issues before. Cutting build times to
1/2 to 1/4 is good.

Checks and tests are harder because often they can't run in parallel. But
then we shouldn't have them listed as independent prerequisites for
targets. Ideally. But fixing it might not be worth the cost since an
"acceptable" work around (rely upon make to serialize the test sequences in
the particular order) is pretty painless (so far).

Of course, having the ability to run the tests 8 at a time (or more) and
reduce the time by 80% would be nice .;-)
On Sep 4, 2012 6:06 PM, "Andrew Dunstan" <andrew(at)dunslane(dot)net> wrote:

>
> On 09/04/2012 05:49 PM, Peter Eisentraut wrote:
>
>> On 9/1/12 12:12 PM, Robert Creager wrote:
>>
>>> I change the build-farm.conf file to have the following make line:
>>>
>>> make => 'make -j 8', # or gmake if required. can include path if
>>> necessary.
>>>
>>> 2 pass, 4 fail. Is this a build configuration you want to pursue?
>>>
>> Sure that would be useful, but it's pretty clear that the check stages
>> don't work in parallel. It think it's because the ports conflict, but
>> there could be any number of other problems.
>>
>> That said, it would be useful, in my mind, to support parallel checks.
>> But unless someone is going to put in the work first, you should
>> restrict your parallel runs to the build and install phases.
>>
>>
>>
>
> The buildfarm code doesn't contain a facility to use a different make
> incantation for each step. It's pretty much an all or nothing deal. Of
> course, you can hack run_build.pl to make it do that, but I highly
> discourage that. For one thing, it makes upgrading that much more
> difficult. All the tweaking is supposed to be done vie the config file. I
> guess I could add a setting that allowed for per step make flags.
>
> Frankly, I have had enough failures of parallel make that I think doing
> this would generate a significant number of non-repeatable failures (I had
> one just the other day that took three invocations of make to get right).
> So I'm not sure doing this would advance us much, although I'm open to
> persuasion.
>
> cheers
>
> andrew
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/**mailpref/pgsql-hackers<http://www.postgresql.org/mailpref/pgsql-hackers>
>
>

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Frankly, I have had enough failures of parallel make that I think doing
> this would generate a significant number of non-repeatable failures (I
> had one just the other day that took three invocations of make to get
> right). So I'm not sure doing this would advance us much, although I'm
> open to persuasion.

Really? I routinely use -j4 for building, and it's been a long time
since I've seen failures. I can believe that for instance "make check"
in contrib would have a problem running in parallel, but the build
process per se seems reliable enough from here.

regards, tom lane

On 09/04/2012 08:37 PM, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> Frankly, I have had enough failures of parallel make that I think doing
>> this would generate a significant number of non-repeatable failures (I
>> had one just the other day that took three invocations of make to get
>> right). So I'm not sure doing this would advance us much, although I'm
>> open to persuasion.
> Really? I routinely use -j4 for building, and it's been a long time
> since I've seen failures. I can believe that for instance "make check"
> in contrib would have a problem running in parallel, but the build
> process per se seems reliable enough from here.
>
>

Both cases were vpath builds, which is what I usually use, if that's a
useful data point.

Maybe I run on lower level hardware than you do. I saw this again this
afternoon after I posted the above. In both cases this was the machine
that runs the buildfarm's crake. I'll try to get a handle on it.

cheers

andrew

On 09/04/2012 08:51 PM, Andrew Dunstan wrote:
>
> On 09/04/2012 08:37 PM, Tom Lane wrote:
>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>> Frankly, I have had enough failures of parallel make that I think doing
>>> this would generate a significant number of non-repeatable failures (I
>>> had one just the other day that took three invocations of make to get
>>> right). So I'm not sure doing this would advance us much, although I'm
>>> open to persuasion.
>> Really? I routinely use -j4 for building, and it's been a long time
>> since I've seen failures. I can believe that for instance "make check"
>> in contrib would have a problem running in parallel, but the build
>> process per se seems reliable enough from here.
>>
>>
>
>
> Both cases were vpath builds, which is what I usually use, if that's a
> useful data point.
>
> Maybe I run on lower level hardware than you do. I saw this again this
> afternoon after I posted the above. In both cases this was the machine
> that runs the buildfarm's crake. I'll try to get a handle on it.
>
>

Well, it looks like it's always failing on ecpg, with preproc.h not
being made in the right order. Here is the last bit of a make log
starting from when it starts on ecpg. This is pretty repeatable.

cheers

andrew

-----

make -C ecpg all
make[3]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg'
make -C include all
make[4]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/include'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/include'
make -C pgtypeslib all
make -C preproc all
make[4]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
numeric.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/numeric.c -MMD
-MP -MF .deps/numeric.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
common.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/common.c -MMD
-MP -MF .deps/common.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
datetime.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/datetime.c -MMD
-MP -MF .deps/datetime.Po
make[4]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/preproc'
make -C ../../../../src/port all
make[5]: Entering directory `/home/pgl/npgl/vpath.testpar/src/port'
make -C ../backend submake-errcodes
make[6]: Entering directory `/home/pgl/npgl/vpath.testpar/src/backend'
make[6]: Nothing to be done for `submake-errcodes'.
make[6]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/backend'
make[5]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/port'
'/usr/bin/perl'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parse.pl
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc <
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y
> preproc.y
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
dt_common.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/dt_common.c -MMD
-MP -MF .deps/dt_common.Po
'/usr/bin/perl'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/check_rules.pl
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/../../../backend/parser/gram.y
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
timestamp.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/timestamp.c -MMD
-MP -MF .deps/timestamp.Po
/usr/bin/flex -o'pgc.c'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/pgc.l
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o type.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/type.c -MMD -MP -MF
.deps/type.Po
rm -f pgstrcasecmp.c && ln -s
/home/andrew/pgl/pg_head/src/port/pgstrcasecmp.c .
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
interval.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/interval.c -MMD
-MP -MF .deps/interval.Po
( echo '{ global:'; gawk '/^[^#]/ {printf "%s;\n",$1}'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/pgtypeslib/exports.txt;
echo ' local: *; };' ) >exports.list
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/include/utils
-I/home/andrew/pgl/pg_head/src/interfaces/libpq
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=3 -c -o
pgstrcasecmp.o pgstrcasecmp.c -MMD -MP -MF .deps/pgstrcasecmp.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o ecpg.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/ecpg.c -MMD -MP -MF
.deps/ecpg.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o output.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/output.c -MMD -MP
-MF .deps/output.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o parser.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parser.c -MMD -MP
-MF .deps/parser.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -DECPG_COMPILE -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include -I.
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc -DMAJOR_VERSION=4
-DMINOR_VERSION=9 -DPATCHLEVEL=0 -I../../../../src/include
-I/home/andrew/pgl/pg_head/src/include -D_GNU_SOURCE
-I/usr/include/libxml2 -c -o keywords.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/keywords.c -MMD -MP
-MF .deps/keywords.Po
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/parser.c:25:21:
fatal error: preproc.h: No such file or directory
compilation terminated.
/home/andrew/pgl/pg_head/src/interfaces/ecpg/preproc/keywords.c:20:21:
fatal error: preproc.h: No such file or directory
compilation terminated.
make[4]: *** [parser.o] Error 1
make[4]: *** Waiting for unfinished jobs....
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -shared -Wl,-soname,libpgtypes.so.3
-Wl,--version-script=exports.list -o libpgtypes.so.3.4 numeric.o
datetime.o common.o dt_common.o timestamp.o interval.o pgstrcasecmp.o
-L../../../../src/port -Wl,--as-needed
-Wl,-rpath,'/home/andrew/pgl/inst.testpar.5701/lib',--enable-new-dtags -lm
ar crs libpgtypes.a numeric.o datetime.o common.o dt_common.o
timestamp.o interval.o pgstrcasecmp.o
ranlib libpgtypes.a
rm -f libpgtypes.so.3
ln -s libpgtypes.so.3.4 libpgtypes.so.3
rm -f libpgtypes.so
ln -s libpgtypes.so.3.4 libpgtypes.so
make[4]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
make -C ecpglib all
make[4]: *** [keywords.o] Error 1
make[4]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/preproc'
make[3]: *** [all-preproc-recurse] Error 2
make[3]: *** Waiting for unfinished jobs....
make[4]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/ecpglib'
make -C ../../../../src/interfaces/libpq all
make -C ../../../../src/interfaces/ecpg/pgtypeslib all
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
execute.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/execute.c
-MMD -MP -MF .deps/execute.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
typename.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/typename.c -MMD -MP
-MF .deps/typename.Po
make[5]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
make[5]: Nothing to be done for `all'.
make[5]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/pgtypeslib'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
descriptor.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/descriptor.c -MMD
-MP -MF .deps/descriptor.Po
make[5]: Entering directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/libpq'
make[5]: Nothing to be done for `all'.
make[5]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/libpq'
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
sqlda.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/sqlda.c
-MMD -MP -MF .deps/sqlda.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o data.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/data.c -MMD -MP -MF
.deps/data.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
error.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/error.c
-MMD -MP -MF .deps/error.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
prepare.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/prepare.c
-MMD -MP -MF .deps/prepare.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
memory.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/memory.c
-MMD -MP -MF .deps/memory.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
connect.o /home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/connect.c
-MMD -MP -MF .deps/connect.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o misc.o
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/misc.c -MMD -MP -MF
.deps/misc.Po
rm -f path.c && ln -s /home/andrew/pgl/pg_head/src/port/path.c .
rm -f pgstrcasecmp.c && ln -s
/home/andrew/pgl/pg_head/src/port/pgstrcasecmp.c .
rm -f strlcpy.c && ln -s /home/andrew/pgl/pg_head/src/port/strlcpy.c .
rm -f thread.c && ln -s /home/andrew/pgl/pg_head/src/port/thread.c .
( echo '{ global:'; gawk '/^[^#]/ {printf "%s;\n",$1}'
/home/andrew/pgl/pg_head/src/interfaces/ecpg/ecpglib/exports.txt; echo '
local: *; };' ) >exports.list
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o path.o
path.c -MMD -MP -MF .deps/path.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
pgstrcasecmp.o pgstrcasecmp.c -MMD -MP -MF .deps/pgstrcasecmp.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
strlcpy.o strlcpy.c -MMD -MP -MF .deps/strlcpy.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -I../include
-I/home/andrew/pgl/pg_head/src/interfaces/ecpg/include
-I/home/andrew/pgl/pg_head/src/interfaces/libpq -I../../../../src/port
-I../../../../src/include -I/home/andrew/pgl/pg_head/src/include
-D_GNU_SOURCE -I/usr/include/libxml2 -DSO_MAJOR_VERSION=6 -c -o
thread.o thread.c -MMD -MP -MF .deps/thread.Po
ccache gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-fexcess-precision=standard -g -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -fpic -shared -Wl,-soname,libecpg.so.6
-Wl,--version-script=exports.list -o libecpg.so.6.5 execute.o typename.o
descriptor.o sqlda.o data.o error.o prepare.o memory.o connect.o misc.o
path.o pgstrcasecmp.o strlcpy.o thread.o -L../../../../src/port
-Wl,--as-needed
-Wl,-rpath,'/home/andrew/pgl/inst.testpar.5701/lib',--enable-new-dtags
-L../pgtypeslib -lpgtypes -L../../../../src/interfaces/libpq -lpq -lm
-lpthread
ar crs libecpg.a execute.o typename.o descriptor.o sqlda.o data.o
error.o prepare.o memory.o connect.o misc.o path.o pgstrcasecmp.o
strlcpy.o thread.o
ranlib libecpg.a
rm -f libecpg.so.6
ln -s libecpg.so.6.5 libecpg.so.6
rm -f libecpg.so
ln -s libecpg.so.6.5 libecpg.so
make[4]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg/ecpglib'
make[3]: Leaving directory
`/home/pgl/npgl/vpath.testpar/src/interfaces/ecpg'
make[2]: *** [all-ecpg-recurse] Error 2
make[2]: Leaving directory `/home/pgl/npgl/vpath.testpar/src/interfaces'
make[1]: *** [all-interfaces-recurse] Error 2
make[1]: Leaving directory `/home/pgl/npgl/vpath.testpar/src'
make: *** [all-src-recurse] Error 2

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Well, it looks like it's always failing on ecpg, with preproc.h not
> being made in the right order. Here is the last bit of a make log
> starting from when it starts on ecpg. This is pretty repeatable.

Hmph. I can't reproduce it at all on my Fedora 16 box. What version
of make are you using?

regards, tom lane

On 09/07/2012 08:43 PM, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> Well, it looks like it's always failing on ecpg, with preproc.h not
>> being made in the right order. Here is the last bit of a make log
>> starting from when it starts on ecpg. This is pretty repeatable.
> Hmph. I can't reproduce it at all on my Fedora 16 box. What version
> of make are you using?

$ make -v
GNU Make 3.82
Built for x86_64-redhat-linux-gnu

cheers

andrew

On 09/07/2012 09:55 PM, Andrew Dunstan wrote:
>
> On 09/07/2012 08:43 PM, Tom Lane wrote:
>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>> Well, it looks like it's always failing on ecpg, with preproc.h not
>>> being made in the right order. Here is the last bit of a make log
>>> starting from when it starts on ecpg. This is pretty repeatable.
>> Hmph. I can't reproduce it at all on my Fedora 16 box. What version
>> of make are you using?
>
>
> $ make -v
> GNU Make 3.82
> Built for x86_64-redhat-linux-gnu
>
>
>

OK, I just tried on a different F16 machine and it didn't happen. I
wonder what's different.

cheers

andrew

On 09/07/2012 10:46 PM, Andrew Dunstan wrote:
>
> On 09/07/2012 09:55 PM, Andrew Dunstan wrote:
>>
>> On 09/07/2012 08:43 PM, Tom Lane wrote:
>>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>>> Well, it looks like it's always failing on ecpg, with preproc.h not
>>>> being made in the right order. Here is the last bit of a make log
>>>> starting from when it starts on ecpg. This is pretty repeatable.
>>> Hmph. I can't reproduce it at all on my Fedora 16 box. What version
>>> of make are you using?
>>
>>
>> $ make -v
>> GNU Make 3.82
>> Built for x86_64-redhat-linux-gnu
>>
>>
>>
>
>
> OK, I just tried on a different F16 machine and it didn't happen. I
> wonder what's different.

This seems totally stupid, but it happens when the path to the current
directory includes a cross-device symlink. If I cd following the link,
then this effect doesn't happen. Weird.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> This seems totally stupid, but it happens when the path to the current
> directory includes a cross-device symlink. If I cd following the link,
> then this effect doesn't happen. Weird.

Huh. So maybe a gmake bug, or maybe there's something wrong with our
make rules for the case that `pwd` doesn't match what gmake thinks the
current path is. Could you specify the setup more fully?

regards, tom lane

On 09/08/2012 11:06 AM, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> This seems totally stupid, but it happens when the path to the current
>> directory includes a cross-device symlink. If I cd following the link,
>> then this effect doesn't happen. Weird.
> Huh. So maybe a gmake bug, or maybe there's something wrong with our
> make rules for the case that `pwd` doesn't match what gmake thinks the
> current path is. Could you specify the setup more fully?
>
>

Scratch that theory, that was just a transient. If anything it looks
like it is related to system load. When almost nothing was running on
the machine it worked fine. When I started up a Browser and an MUA the
problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
average around 0.4 right now. I'm a bit perplexed.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Scratch that theory, that was just a transient. If anything it looks
> like it is related to system load. When almost nothing was running on
> the machine it worked fine. When I started up a Browser and an MUA the
> problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
> average around 0.4 right now. I'm a bit perplexed.

Hm ... you weren't using the -l (--max-load) option were you? That
would make system load affect gmake's scheduling. Although it's clear
when I test it that it is waiting for the bison run to finish before
launching the dependent builds, so it still seems like it must be a bug
if your copy isn't waiting for that.

regards, tom lane

On 09/08/2012 04:46 PM, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> Scratch that theory, that was just a transient. If anything it looks
>> like it is related to system load. When almost nothing was running on
>> the machine it worked fine. When I started up a Browser and an MUA the
>> problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
>> average around 0.4 right now. I'm a bit perplexed.
> Hm ... you weren't using the -l (--max-load) option were you? That
> would make system load affect gmake's scheduling. Although it's clear
> when I test it that it is waiting for the bison run to finish before
> launching the dependent builds, so it still seems like it must be a bug
> if your copy isn't waiting for that.
>
>

No. just "make -j 4"

And it's the stock Fedora build of make.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> And it's the stock Fedora build of make.

Which Fedora branch exactly?

The package version I was trying to reproduce with here is
make-3.82-8.fc16.x86_64.

regards, tom lane

On 09/08/2012 04:54 PM, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> And it's the stock Fedora build of make.
> Which Fedora branch exactly?
>
> The package version I was trying to reproduce with here is
> make-3.82-8.fc16.x86_64.

Same:

$ rpm -q make
make-3.82-8.fc16.x86_64

cheers

andrew

On 09/08/2012 04:52 PM, Andrew Dunstan wrote:
>
> On 09/08/2012 04:46 PM, Tom Lane wrote:
>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>> Scratch that theory, that was just a transient. If anything it looks
>>> like it is related to system load. When almost nothing was running on
>>> the machine it worked fine. When I started up a Browser and an MUA the
>>> problem occurred. This VM has 4 CPUs and 4Gb of memory and a load
>>> average around 0.4 right now. I'm a bit perplexed.
>> Hm ... you weren't using the -l (--max-load) option were you? That
>> would make system load affect gmake's scheduling. Although it's clear
>> when I test it that it is waiting for the bison run to finish before
>> launching the dependent builds, so it still seems like it must be a bug
>> if your copy isn't waiting for that.
>>
>>
>
> No. just "make -j 4"
>
> And it's the stock Fedora build of make.

I have just repeated this on an absolutely fresh up to date F17 machine,
with no symlink stuff in play.

Steps to recreate:

CC="ccache gcc" ../postgres/configure --enable-depend --enable-debug
--enable-cassert --with-perl --with-python --with-tcl --with-libxml
--with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap
make -j 4

cheers

andrew

q

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> I have just repeated this on an absolutely fresh up to date F17 machine,
> with no symlink stuff in play.

> Steps to recreate:

> CC="ccache gcc" ../postgres/configure --enable-depend --enable-debug
> --enable-cassert --with-perl --with-python --with-tcl --with-libxml
> --with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap
> make -j 4

Huh ... that recipe works (er, fails) for me too, at least some of the
time. I wonder what exactly is the key difference between the working
and failing cases?

Anyway, what I notice is that I get different types of failures, but
they are all under ecpg/. What I think we need to do is insert
.NOTPARALLEL in ecpg/Makefile, because there are several reasons not
to run its sub-makes in parallel:

* preproc/Makefile casually does this:

../ecpglib/typename.o: ../ecpglib/typename.c
$(MAKE) -C $(dir $@) $(notdir $@)

which is very likely to screw up any make proceeding in parallel in
ecpglib.

* compatlib and ecpglib will equally casually invoke "make all" in
other subdirectories; ditto.

And that's not even counting the bison-output problem you were seeing.
I'm not entirely sure what's causing that, but I'm suspicious that the
ultimate cause is the extra rules for the "all...recurse" targets in
ecpg/Makefile, which look like they could result in additional instances
of multiple make processes running in the same subdirectory.

After adding the .NOTPARALLEL marker, I don't see these failures
anymore.

regards, tom lane

On 09/08/2012 07:54 PM, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> I have just repeated this on an absolutely fresh up to date F17 machine,
>> with no symlink stuff in play.
>> Steps to recreate:
>> CC="ccache gcc" ../postgres/configure --enable-depend --enable-debug
>> --enable-cassert --with-perl --with-python --with-tcl --with-libxml
>> --with-libxslt -with-openssl --with-gssapi --with-pam --with-ldap
>> make -j 4
> Huh ... that recipe works (er, fails) for me too, at least some of the
> time. I wonder what exactly is the key difference between the working
> and failing cases?
>
> Anyway, what I notice is that I get different types of failures, but
> they are all under ecpg/. What I think we need to do is insert
> .NOTPARALLEL in ecpg/Makefile, because there are several reasons not
> to run its sub-makes in parallel:
>
> * preproc/Makefile casually does this:
>
> ../ecpglib/typename.o: ../ecpglib/typename.c
> $(MAKE) -C $(dir $@) $(notdir $@)
>
> which is very likely to screw up any make proceeding in parallel in
> ecpglib.
>
> * compatlib and ecpglib will equally casually invoke "make all" in
> other subdirectories; ditto.
>
> And that's not even counting the bison-output problem you were seeing.
> I'm not entirely sure what's causing that, but I'm suspicious that the
> ultimate cause is the extra rules for the "all...recurse" targets in
> ecpg/Makefile, which look like they could result in additional instances
> of multiple make processes running in the same subdirectory.
>
> After adding the .NOTPARALLEL marker, I don't see these failures
> anymore.
>
>

Well, I'm glad it's not just me. :-)

This fix works for me too.

I guess it should be applied back to 9.1, when it looks like we started
using .NOTPARALLEL

cheers

andrew

On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
> Anyway, what I notice is that I get different types of failures, but
> they are all under ecpg/. What I think we need to do is insert
> .NOTPARALLEL in ecpg/Makefile,

I'd hate that, because the ecpg build is one of the slowest parts of the
build, so de-parallelizing it would slow down everything quite a bit.

> because there are several reasons not
> to run its sub-makes in parallel:
>
> * preproc/Makefile casually does this:
>
> ../ecpglib/typename.o: ../ecpglib/typename.c
> $(MAKE) -C $(dir $@) $(notdir $@)
>
> which is very likely to screw up any make proceeding in parallel in
> ecpglib.

That should probably be fixed by symlinking the source file and building
it in the preproc directory.

> And that's not even counting the bison-output problem you were seeing.
> I'm not entirely sure what's causing that, but I'm suspicious that the
> ultimate cause is the extra rules for the "all...recurse" targets in
> ecpg/Makefile, which look like they could result in additional instances
> of multiple make processes running in the same subdirectory.

I think the point of these targets is exactly to prevent that.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
>> Anyway, what I notice is that I get different types of failures, but
>> they are all under ecpg/. What I think we need to do is insert
>> .NOTPARALLEL in ecpg/Makefile,

> I'd hate that, because the ecpg build is one of the slowest parts of the
> build, so de-parallelizing it would slow down everything quite a bit.

There's only one bit of it that's slow, which is the bison build +
preproc.c compile, which is necessarily serial anyway. So I think
trying to avoid .NOTPARALLEL there is a complete waste of effort.
But if you wanna fix it some other way, step right up.

(This all does remind me of the "Recursive Make Considered Harmful"
thread from awhile back. Is anyone prepared to go down that path
now? I'm not terribly excited by that, as yet --- yeah, it would
be better, but the work involved seems out of proportion to the
benefit.)

regards, tom lane

On 09/09/2012 03:29 AM, Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>> On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
>>> Anyway, what I notice is that I get different types of failures, but
>>> they are all under ecpg/. What I think we need to do is insert
>>> .NOTPARALLEL in ecpg/Makefile,
>> I'd hate that, because the ecpg build is one of the slowest parts of the
>> build, so de-parallelizing it would slow down everything quite a bit.
> There's only one bit of it that's slow, which is the bison build +
> preproc.c compile, which is necessarily serial anyway. So I think
> trying to avoid .NOTPARALLEL there is a complete waste of effort.
> But if you wanna fix it some other way, step right up.

Yeah. I am going to add a config parameter to the buildfarm to allow
parallelism for the "make" and "make contrib" stages, but I'm not going
to release it until this is fixed.

(I suppose this is also a lesson to me that I should not ignore things
like this that annoy me persistently as I did with these failures until
Robert Creager started this discussion. It just didn't seem important
enough.)

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> On 09/09/2012 03:29 AM, Tom Lane wrote:
>> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>>> On Sat, 2012-09-08 at 19:54 -0400, Tom Lane wrote:
>>>> Anyway, what I notice is that I get different types of failures, but
>>>> they are all under ecpg/. What I think we need to do is insert
>>>> .NOTPARALLEL in ecpg/Makefile,

>>> I'd hate that, because the ecpg build is one of the slowest parts of the
>>> build, so de-parallelizing it would slow down everything quite a bit.

>> There's only one bit of it that's slow, which is the bison build +
>> preproc.c compile, which is necessarily serial anyway. So I think
>> trying to avoid .NOTPARALLEL there is a complete waste of effort.
>> But if you wanna fix it some other way, step right up.

> Yeah. I am going to add a config parameter to the buildfarm to allow
> parallelism for the "make" and "make contrib" stages, but I'm not going
> to release it until this is fixed.

Well, why don't we stick .NOTPARALLEL in there for the moment, and then
if Peter thinks of a better solution, he can revert that change in favor
of something cleaner.

I assume we need this for all active branches, if the buildfarm is
going to be stressing it?

regards, tom lane

On 09/09/2012 11:31 AM, Tom Lane wrote:
>> Yeah. I am going to add a config parameter to the buildfarm to allow
>> parallelism for the "make" and "make contrib" stages, but I'm not going
>> to release it until this is fixed.
> Well, why don't we stick .NOTPARALLEL in there for the moment, and then
> if Peter thinks of a better solution, he can revert that change in favor
> of something cleaner.

Works for me

>
> I assume we need this for all active branches, if the buildfarm is
> going to be stressing it?
>
>

I can restrict it to only modern branches. Didn't we supposedly improve
support for this during the 9.1 cycle? That seems like a sensible place
to go back to.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> On 09/09/2012 11:31 AM, Tom Lane wrote:
>> I assume we need this for all active branches, if the buildfarm is
>> going to be stressing it?

> I can restrict it to only modern branches. Didn't we supposedly improve
> support for this during the 9.1 cycle? That seems like a sensible place
> to go back to.

Mmm, yeah, the commit history shows several different fixes for parallel
make back in the 9.1 time frame. I doubt we want to try back-porting
all that work. Let's just say 9.1 and up, then.

regards, tom lane

And the answer is ... it's a gmake bug. Apparently introduced in 3.82.

http://savannah.gnu.org/bugs/?30653
https://bugzilla.redhat.com/show_bug.cgi?id=835424

So I think .NOTPARALLEL is just masking the true problem, but
nonetheless it's a problem. And given that the bug report on savannah
has been ignored for two years, we should not hold our breath for a fix
to appear upstream (much less propagate to everyone using 3.82).

regards, tom lane

On 09/09/2012 02:05 PM, Tom Lane wrote:
> And the answer is ... it's a gmake bug. Apparently introduced in 3.82.
>
> http://savannah.gnu.org/bugs/?30653
> https://bugzilla.redhat.com/show_bug.cgi?id=835424
>
> So I think .NOTPARALLEL is just masking the true problem, but
> nonetheless it's a problem. And given that the bug report on savannah
> has been ignored for two years, we should not hold our breath for a fix
> to appear upstream (much less propagate to everyone using 3.82).
>
>

Thanks for pursuing this. Whether or not it masks the underlying
problem, it's still something we should do, no? In fact, it seems to me
like this makes it even less worth trying to do anything better.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> On 09/09/2012 02:05 PM, Tom Lane wrote:
>> And the answer is ... it's a gmake bug.

> Thanks for pursuing this. Whether or not it masks the underlying
> problem, it's still something we should do, no? In fact, it seems to me
> like this makes it even less worth trying to do anything better.

Yeah, exactly.

regards, tom lane

On Sun, 2012-09-09 at 14:05 -0400, Tom Lane wrote:
> And the answer is ... it's a gmake bug. Apparently introduced in 3.82.
>
> http://savannah.gnu.org/bugs/?30653
> https://bugzilla.redhat.com/show_bug.cgi?id=835424
>
> So I think .NOTPARALLEL is just masking the true problem, but
> nonetheless it's a problem. And given that the bug report on savannah
> has been ignored for two years, we should not hold our breath for a fix
> to appear upstream (much less propagate to everyone using 3.82).

So does the issue go away when using GNU make 3.81?

On Sun, 2012-09-09 at 14:57 -0400, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> > On 09/09/2012 02:05 PM, Tom Lane wrote:
> >> And the answer is ... it's a gmake bug.
>
> > Thanks for pursuing this. Whether or not it masks the underlying
> > problem, it's still something we should do, no? In fact, it seems to me
> > like this makes it even less worth trying to do anything better.
>
> Yeah, exactly.

But then the answer could be, if you want to use parallel make, use a
version that's not broken.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> On Sun, 2012-09-09 at 14:05 -0400, Tom Lane wrote:
>> So I think .NOTPARALLEL is just masking the true problem, but
>> nonetheless it's a problem. And given that the bug report on savannah
>> has been ignored for two years, we should not hold our breath for a fix
>> to appear upstream (much less propagate to everyone using 3.82).

> So does the issue go away when using GNU make 3.81?

I couldn't reproduce it in a few tries on my Mac laptop, which has 3.81.
But that's not necessarily a comparable test, given the different
hardware, OS, and compiler version. I'd put more faith in the multiple
statements in the upstream bugs that people weren't seeing this with
3.81.

regards, tom lane

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> But then the answer could be, if you want to use parallel make, use a
> version that's not broken.

That's not a terribly practical answer for people who use the "make"
supplied by their OS vendor, which is approximately 99.9% of people.
It's even less practical for packagers, who don't have a choice about
what tool set to use.

Even if I wanted to use a locally-patched make, I'm not sure I'd trust a
patch that doesn't seem to have been signed off on by any actual gmake
developer or maintainer. That sort of cure is frequently worse than the
disease.

regards, tom lane

On 09/09/2012 05:00 PM, Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>> But then the answer could be, if you want to use parallel make, use a
>> version that's not broken.
> That's not a terribly practical answer for people who use the "make"
> supplied by their OS vendor, which is approximately 99.9% of people.
> It's even less practical for packagers, who don't have a choice about
> what tool set to use.
>
> Even if I wanted to use a locally-patched make, I'm not sure I'd trust a
> patch that doesn't seem to have been signed off on by any actual gmake
> developer or maintainer. That sort of cure is frequently worse than the
> disease.
>
>

Right.

Meanwhile, I have buildfarm animal crake building with the experimental
buildfarm code, and so far the results look good.

cheers

andrew

I wrote:
> And the answer is ... it's a gmake bug. Apparently introduced in 3.82.
> http://savannah.gnu.org/bugs/?30653
> https://bugzilla.redhat.com/show_bug.cgi?id=835424

> So I think .NOTPARALLEL is just masking the true problem, but
> nonetheless it's a problem. And given that the bug report on savannah
> has been ignored for two years, we should not hold our breath for a fix
> to appear upstream (much less propagate to everyone using 3.82).

So no sooner do I complain about that, than the upstream maintainers
wake up and commit it:
http://lists.gnu.org/archive/html/bug-make/2012-09/msg00016.html

No idea when a fixed release might appear, but at least somebody who
knows the gmake code has signed off on the fix now.

regards, tom lane

On 09/10/2012 02:44 PM, Tom Lane wrote:
> I wrote:
>> And the answer is ... it's a gmake bug. Apparently introduced in 3.82.
>> http://savannah.gnu.org/bugs/?30653
>> https://bugzilla.redhat.com/show_bug.cgi?id=835424
>> So I think .NOTPARALLEL is just masking the true problem, but
>> nonetheless it's a problem. And given that the bug report on savannah
>> has been ignored for two years, we should not hold our breath for a fix
>> to appear upstream (much less propagate to everyone using 3.82).
> So no sooner do I complain about that, than the upstream maintainers
> wake up and commit it:
> http://lists.gnu.org/archive/html/bug-make/2012-09/msg00016.html
>
> No idea when a fixed release might appear, but at least somebody who
> knows the gmake code has signed off on the fix now.
>
>

When it does appear in a release I guess we could make the .NOTPARALLEL
conditional on make version. If not, we'll have to wait a long time
before removing it.

cheers

andrew