Re: bin checks taking too long.

Currently the buildfarm animal crake (my development instance) is
running the bin check, but not any other animal. These tests still take
for too long, not least because each set of tests requires a separate
install.

I have complained about this before, but we don't seem to have made any
progress. I am very reluctant to put this check into a buildfarm client
release until that is addressed. Is there really nothing we can do about
it? This is getting more important since we now have another feature
that we want to get into a buildfarm client release.

cheers

andrew

On 12/22/14 7:56 PM, Andrew Dunstan wrote:
> Currently the buildfarm animal crake (my development instance) is
> running the bin check, but not any other animal. These tests still take
> for too long, not least because each set of tests requires a separate
> install.

You can avoid that by using make installcheck.

> Is there really nothing we can do about it?

There is, but it's not straightforward, as it turns out. Ideas are welcome.

Also, as you can imagine, any build system changes are bottlenecked on
Windows support.

On 12/22/2014 10:41 PM, Peter Eisentraut wrote:
> On 12/22/14 7:56 PM, Andrew Dunstan wrote:
>> Currently the buildfarm animal crake (my development instance) is
>> running the bin check, but not any other animal. These tests still take
>> for too long, not least because each set of tests requires a separate
>> install.
> You can avoid that by using make installcheck.

Not working too well:

make -C pg_ctl installcheck
make[1]: Entering directory
`/home/bf/bfr/root/HEAD/pgsql.build/src/bin/pg_ctl'
cd /home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/bin/pg_ctl &&
TESTDIR='/home/bf/bfr/root/HEAD/pgsql.build/src/bin/pg_ctl'
PATH="/home/bf/bfr/root/HEAD/inst/bin:$PATH" PGPORT='65648' prove -I
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/ --verbose
t/*.pl
Use of uninitialized value $ENV{"top_builddir"} in concatenation (.)
or string at t/001_start_stop.pl line 17.
file not found: /src/test/regress/pg_regress at
/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/TestLib.pm
line 142.

This was invoked like this:

cd $pgsql/src/bin && make NO_LOCALE=1 installcheck

Note that it's a vpath build.

cheers

andrew

On 12/23/2014 11:11 AM, Andrew Dunstan wrote:
>
> On 12/22/2014 10:41 PM, Peter Eisentraut wrote:
>> On 12/22/14 7:56 PM, Andrew Dunstan wrote:
>>> Currently the buildfarm animal crake (my development instance) is
>>> running the bin check, but not any other animal. These tests still take
>>> for too long, not least because each set of tests requires a separate
>>> install.
>> You can avoid that by using make installcheck.
>
>
> Not working too well:
>
> make -C pg_ctl installcheck
> make[1]: Entering directory
> `/home/bf/bfr/root/HEAD/pgsql.build/src/bin/pg_ctl'
> cd /home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/bin/pg_ctl &&
> TESTDIR='/home/bf/bfr/root/HEAD/pgsql.build/src/bin/pg_ctl'
> PATH="/home/bf/bfr/root/HEAD/inst/bin:$PATH" PGPORT='65648' prove -I
> /home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/ --verbose
> t/*.pl
> Use of uninitialized value $ENV{"top_builddir"} in concatenation (.)
> or string at t/001_start_stop.pl line 17.
> file not found: /src/test/regress/pg_regress at
> /home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/TestLib.pm
> line 142.
>
> This was invoked like this:
>
> cd $pgsql/src/bin && make NO_LOCALE=1 installcheck
>
> Note that it's a vpath build.
>
>

The attached patch seems to fix this. However, running installcheck also
makes very little difference to the time. These checks still take around
100 seconds on my machine, including around 60 for the script tests.
That seems ridiculously long, when the whole core regression suite takes
23 seconds.

cheers

andrew

On Tue, Dec 23, 2014 at 01:55:46PM -0500, Andrew Dunstan wrote:
> On 12/23/2014 11:11 AM, Andrew Dunstan wrote:
> > Use of uninitialized value $ENV{"top_builddir"} in concatenation (.)
> > or string at t/001_start_stop.pl line 17.
> > file not found: /src/test/regress/pg_regress at
> >/home/bf/bfr/root/HEAD/pgsql.build/../pgsql/src/test/perl/TestLib.pm
> > line 142.
> >
> >This was invoked like this:
> >
> > cd $pgsql/src/bin && make NO_LOCALE=1 installcheck
> >
> >Note that it's a vpath build.
> >
> >
>
> The attached patch seems to fix this.

Oops. Your fix looks good.

> However, running installcheck also
> makes very little difference to the time. These checks still take around 100
> seconds on my machine, including around 60 for the script tests. That seems
> ridiculously long, when the whole core regression suite takes 23 seconds.

I am satisfied with the performance of the TAP suites.