Re: Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

On Sun, Mar 18, 2012 at 7:35 PM, Peter Geoghegan <peter(at)2ndquadrant(dot)com> wrote:
> On 19 March 2012 01:50, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I am *not* a fan of regression tests that try to microscopically test
>> every feature in the system.
>
> I see your point of view. I suppose I can privately hold onto the test
> suite, since it might prove useful again.
>
> I will work on a pg_regress based approach with a reasonably-sized
> random subset of about 20 of my existing tests, to provide some basic
> smoke testing.

This may sound rather tortured, but in the main regression suite there
is a .c file that links some stuff into the backend that is then
accessed via CREATE FUNCTION to do some special fiddly bits. Could a
creative hook be used here to avoid the repetition you are avoiding
via Python? (e.g. constant resetting of pg_stat_statements or
whatnot). It might sound too much like changing the system under
test, but I think it would still retain most of the value.

I also do like the pg_regress workflow in general, although clearly it
cannot do absolutely everything. Running and interpreting the results
of your tests was not hard, but it was definitely *different* which
could be a headache if one-off testing frameworks proliferate.

--
fdr