Re: parallel regression test output

There is something slightly funny about the regression test output.
When it starts a parallel group, it prints something like

parallel group (14 tests): select_views portals_p2 rules foreign_key
cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data
window xmlmap

One would think that it prints this as the tests are started, but it
actually prints it as the tests finish. The last one or two test names
are always printed after a considerable delay, probably because of
terminal timing issues, which I have found confusing on several
occasions.

Note also that that behavior is inconsistent with that of a serial test
like

test misc ... ok

Here, the first part of the line is printed before that test starts, and
the result is printed after.

I propose to adjust the behavior so that the "parallel group" line
prints the test names before/as the tests are started, and then the
subsequent lines print the results after the tests finish, as is
currently the case.

See attached patch.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> I propose to adjust the behavior so that the "parallel group" line
> prints the test names before/as the tests are started, and then the
> subsequent lines print the results after the tests finish, as is
> currently the case.

This is not actually going to work nicely in cases such as the number of
parallel tests being limited by max_connections. Personally I never
found the current behavior objectionable ...

regards, tom lane

I wrote:
> This is not actually going to work nicely in cases such as the number of
> parallel tests being limited by max_connections.

Er, no, I take that back: I was thinking that the status() function
included checking of the output file, but it only prints the test name.

Still, I'm not convinced this is an improvement. The current behavior
gives some feedback about how the set of parallel tests is progressing,
while what you suggest would typically just print all the test names
right away and then sit there for a long time.

regards, tom lane

On Sun, Jan 17, 2010 at 3:19 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> There is something slightly funny about the regression test output.
> When it starts a parallel group, it prints something like
>
> parallel group (14 tests):  select_views portals_p2 rules foreign_key
> cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data
> window xmlmap
>
> One would think that it prints this as the tests are started, but it
> actually prints it as the tests finish.

I always assumed that it was printing the names as the tests finished,
probably because of the way the output is staggered. If they were
printed when the tests were started, the list would be printed all but
simultaneously.

> The last one or two test names
> are always printed after a considerable delay, probably because of
> terminal timing issues, which I have found confusing on several
> occasions.

Terminal timing issues?

> I propose to adjust the behavior so that the "parallel group" line
> prints the test names before/as the tests are started, and then the
> subsequent lines print the results after the tests finish, as is
> currently the case.

I think I prefer the current behavior. As Tom said downthread, it
gives you a sort of progress indicator.

...Robert

On sön, 2010-01-17 at 23:54 -0500, Robert Haas wrote:
> I always assumed that it was printing the names as the tests finished,
> probably because of the way the output is staggered. If they were
> printed when the tests were started, the list would be printed all but
> simultaneously.

Well, as long as someone expects it to be that way, then I guess that'd
be OK. I am concerned about an inconsistency with the serial tests that
go

test foo ... <pause>ok

whereas with the parallel tests don't give any indication what test has
started.

On Mon, Jan 18, 2010 at 4:54 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On sön, 2010-01-17 at 23:54 -0500, Robert Haas wrote:
>> I always assumed that it was printing the names as the tests finished,
>> probably because of the way the output is staggered.  If they were
>> printed when the tests were started, the list would be printed all but
>> simultaneously.
>
> Well, as long as someone expects it to be that way, then I guess that'd
> be OK.  I am concerned about an inconsistency with the serial tests that
> go
>
> test foo ... <pause>ok
>
> whereas with the parallel tests don't give any indication what test has
> started.

I always thought of the words "parallel group (x tests)" as being
parallel to printing "test foo ..." -- though admittedly that doesn't
give you specific information about which tests are starting at which
time. We certainly could design some system where we signal
beginnings and ends of tests, but I'm not sure it's worth cluttering
the output. Ordinarily I suspect they all go so fast that it doesn't
matter much.

...Robert