Re: build farm machine using <make -j 8> mixed results

On Sep 4, 2012 6:06 PM, "Andrew Dunstan" <andrew(at)dunslane(dot)net> wrote:
>
>
> Frankly, I have had enough failures of parallel make that I think doing
this would generate a significant number of non-repeatable failures (I had
one just the other day that took three invocations of make to get right).
So I'm not sure doing this would advance us much, although I'm open to
persuasion.

Seeing as most PostgreSQL bugs appear with concurrency, I think we should
leave our default config with 1 for max connections.

;-)

Parallel make failures are bugs in the dependencies as described in our
make files.

For the build phase, I don't recall parallel problems and as a habit I
usually use parallel makes. I would like that to be supported and I think
I've seen fixes applied when we had issues before. Cutting build times to
1/2 to 1/4 is good.

Checks and tests are harder because often they can't run in parallel. But
then we shouldn't have them listed as independent prerequisites for
targets. Ideally. But fixing it might not be worth the cost since an
"acceptable" work around (rely upon make to serialize the test sequences in
the particular order) is pretty painless (so far).

Of course, having the ability to run the tests 8 at a time (or more) and
reduce the time by 80% would be nice .;-)
On Sep 4, 2012 6:06 PM, "Andrew Dunstan" <andrew(at)dunslane(dot)net> wrote:

>
> On 09/04/2012 05:49 PM, Peter Eisentraut wrote:
>
>> On 9/1/12 12:12 PM, Robert Creager wrote:
>>
>>> I change the build-farm.conf file to have the following make line:
>>>
>>> make => 'make -j 8', # or gmake if required. can include path if
>>> necessary.
>>>
>>> 2 pass, 4 fail. Is this a build configuration you want to pursue?
>>>
>> Sure that would be useful, but it's pretty clear that the check stages
>> don't work in parallel. It think it's because the ports conflict, but
>> there could be any number of other problems.
>>
>> That said, it would be useful, in my mind, to support parallel checks.
>> But unless someone is going to put in the work first, you should
>> restrict your parallel runs to the build and install phases.
>>
>>
>>
>
> The buildfarm code doesn't contain a facility to use a different make
> incantation for each step. It's pretty much an all or nothing deal. Of
> course, you can hack run_build.pl to make it do that, but I highly
> discourage that. For one thing, it makes upgrading that much more
> difficult. All the tweaking is supposed to be done vie the config file. I
> guess I could add a setting that allowed for per step make flags.
>
> Frankly, I have had enough failures of parallel make that I think doing
> this would generate a significant number of non-repeatable failures (I had
> one just the other day that took three invocations of make to get right).
> So I'm not sure doing this would advance us much, although I'm open to
> persuasion.
>
> cheers
>
> andrew
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/**mailpref/pgsql-hackers<http://www.postgresql.org/mailpref/pgsql-hackers>
>
>