Re: Regress tests to improve the function coverage of schemacmds and user and tablespace files

On Sun, Nov 24, 2013 at 6:37 AM, Kevin Grittner <kgrittn(at)ymail(dot)com> wrote:

> David Rowley <dgrowleyml(at)gmail(dot)com> wrote:
>
> > I've just had a look at both of these patches. All tests that
> > have been added seem to cover new areas that are not previously
> > tested, they also seem to cleanup properly after themselves, so I
> > think these should be a worthwhile addition to the regression
> > tests.
>
> Thanks for reviewing! Did you happen to note the impact on `make
> check` runtime? There are many people who run that many times per
> day while working on development, so we try to keep new tests that
> significantly extend that separate. We haven't quite worked out
> the best way to exercise such longer-running tests, but I suspect
> we soon will. At any rate, this is a piece of information the
> committer will want, so you will be helping whoever that is if you
> can supply it.
>
>
I've done a quick benchmark on this this morning.
Note that I'm using windows here and I used powershell to time the
regression run with the following command:

PS D:\Postgres\b\src\tools\msvc> Measure-Command { .\vcregress.bat check }

I ran the tests 10 times each.
I ran the patched version first, then just did git reset --hard to revert
the patched changes then I ran the tests again.

The average and median results over the 10 runs are as follows:

Patched Unpatched Time increased by
Average 48.23265888 47.70979854 101.10%
Median 47.8993686 47.51177815 100.82%

The slowdown is not too bad. It just around 1% increase of time.

I've attached the results in spreadsheet format.

Regards

David Rowley

--
> Kevin Grittner
> EDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>