| Lists: | pgsql-hackers |
|---|
| From: | Andres Freund <andres(at)2ndquadrant(dot)com> |
|---|---|
| To: | pgsql-hackers(at)postgresql(dot)org |
| Subject: | Add .NOTPARALLEL to contrib/Makefile |
| Date: | 2014-08-26 00:05:43 |
| Message-ID: | 20140826000543.GC21544@awork2.anarazel.de |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Lists: | pgsql-hackers |
Hi,
Currently running "make -j16 all check" in contrib/ results in a mess because
all pg_regress invocations fight over the same port. Adding a simple
.NOTPARALLEL: check-%-recurse
into contrib/Makefile fixes that. Do we want that?
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Andres Freund <andres(at)2ndquadrant(dot)com> |
| Cc: | pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: Add .NOTPARALLEL to contrib/Makefile |
| Date: | 2014-08-26 00:16:50 |
| Message-ID: | 24650.1409012210@sss.pgh.pa.us |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Lists: | pgsql-hackers |
Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> Currently running "make -j16 all check" in contrib/ results in a mess because
> all pg_regress invocations fight over the same port. Adding a simple
> .NOTPARALLEL: check-%-recurse
> into contrib/Makefile fixes that. Do we want that?
Dunno, but if we do, it should be applied to installcheck as well.
(In that case the fight is over the contrib_regression database.)
A larger point is that you shouldn't really fix this only for contrib.
What about tests run in src/pl, or contrib vs the main regression tests
vs src/pl vs test/isolationtester, etc etc.
regards, tom lane
| From: | Andres Freund <andres(at)2ndquadrant(dot)com> |
|---|---|
| To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
| Cc: | pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: Add .NOTPARALLEL to contrib/Makefile |
| Date: | 2014-08-26 00:57:19 |
| Message-ID: | 20140826005719.GD21544@awork2.anarazel.de |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Lists: | pgsql-hackers |
On 2014-08-25 20:16:50 -0400, Tom Lane wrote:
> Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> > Currently running "make -j16 all check" in contrib/ results in a mess because
> > all pg_regress invocations fight over the same port. Adding a simple
> > .NOTPARALLEL: check-%-recurse
> > into contrib/Makefile fixes that. Do we want that?
>
> Dunno, but if we do, it should be applied to installcheck as well.
> (In that case the fight is over the contrib_regression database.)
Right. Although you can mostly fight it there using USE_MODULE_DB.
The attached patch that replaces all hardcoded occurrences of
'contrib_regression' with current_database(). Allowing a make -j32 -s
installcheck in contrib to succeed in less than 4 seconds...
That's not particularly pretty, especially in the CREATE SERVER calls
(via DO ... EXECUTE), but imo worth it given the timesavings.
What's your thought on that one?
> A larger point is that you shouldn't really fix this only for contrib.
> What about tests run in src/pl, or contrib vs the main regression tests
> vs src/pl vs test/isolationtester, etc etc.
Unfortunately I don't think you can make .NOTPARALLEL work across more
than one directory when using recursive make :(. At least I don't know
how.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
| Attachment | Content-Type | Size |
|---|---|---|
| 0001-Don-t-hardcode-contrib_regression-dbname-in-postgres.patch | text/x-patch | 14.0 KB |
| From: | Peter Eisentraut <peter_e(at)gmx(dot)net> |
|---|---|
| To: | Andres Freund <andres(at)2ndquadrant(dot)com> |
| Cc: | pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: Add .NOTPARALLEL to contrib/Makefile |
| Date: | 2014-08-27 03:56:10 |
| Message-ID: | 1409111770.13799.12.camel@vanquo.pezone.net |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Lists: | pgsql-hackers |
On Tue, 2014-08-26 at 02:05 +0200, Andres Freund wrote:
> Currently running "make -j16 all check" in contrib/ results in a mess
> because
> all pg_regress invocations fight over the same port. Adding a simple
> .NOTPARALLEL: check-%-recurse
> into contrib/Makefile fixes that. Do we want that?
But that causes also the "all" to be run in not-parallel, because the
meaning of .NOTPARALLEL is:
If `.NOTPARALLEL' is mentioned as a target, then this invocation of
`make' will be run serially, even if the `-j' option is given.
It does not mean, as you appear to imagine, to run only the listed
prerequisites in not-parallel. That would be nice!
| From: | Andres Freund <andres(at)2ndquadrant(dot)com> |
|---|---|
| To: | Peter Eisentraut <peter_e(at)gmx(dot)net> |
| Cc: | pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: Add .NOTPARALLEL to contrib/Makefile |
| Date: | 2014-08-28 11:19:20 |
| Message-ID: | 20140828111920.GC17329@awork2.anarazel.de |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Lists: | pgsql-hackers |
On 2014-08-26 23:56:10 -0400, Peter Eisentraut wrote:
> On Tue, 2014-08-26 at 02:05 +0200, Andres Freund wrote:
> > Currently running "make -j16 all check" in contrib/ results in a mess
> > because
> > all pg_regress invocations fight over the same port. Adding a simple
> > .NOTPARALLEL: check-%-recurse
> > into contrib/Makefile fixes that. Do we want that?
>
> But that causes also the "all" to be run in not-parallel, because the
> meaning of .NOTPARALLEL is:
>
> If `.NOTPARALLEL' is mentioned as a target, then this invocation of
> `make' will be run serially, even if the `-j' option is given.
Argh. There goes that.
> It does not mean, as you appear to imagine, to run only the listed
> prerequisites in not-parallel. That would be nice!
Yea.
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services