Re: performance-test farm

From: Tomas Vondra <tv(at)fuzzy(dot)cz>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: performance-test farm
Date: 2011-05-12 19:01:42
Message-ID: 4DCC2E96.2000103@fuzzy.cz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dne 12.5.2011 16:36, Tom Lane napsal(a):
>> * Josh Berkus (josh(at)agliodbs(dot)com) wrote:
>>> The first problem is plaform performance, which would be a matter of
>>> expanding the buildfarm to include a small set of performance tests ...
>>> probably ones based on previously known problems, plus some other simple
>>> common operations. The goal here would be to test on as many different
>>> machines as possible, rather than getting full coverage of peformance.
>
> I think it's a seriously *bad* idea to expect existing buildfarm members
> to produce useful performance data. Very few of them are running on
> dedicated machines, and some are deliberately configured with
> performance-trashing options. (I think just about all of 'em use
> --enable-cassert, but there are some with worse things...)
>
> We can probably share a great deal of the existing buildfarm code and
> infrastructure, but the actual members of the p-farm will need to be a
> separate collection of machines running different builds.

Yes, I agree that to get reliable and useful performance data, we need
well defined environment (dedicated machines, proper settings) and this
probably is not possible with most of the current buildfarm members.
That's not an issue of the buildfarm - it simply serves other purposes,
not performance testing.

But I believe using the buildfarm framework is a good idea, and maybe we
could even use some of the nodes (those that are properly set).

Not sure if there should be two separate farms (one to test builds, one
to test performance), or if we could run one farm and enable
'performance-testing modules' only for some of the members.

regards
Tomas

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2011-05-12 19:17:04 Re: performance-test farm
Previous Message Kevin Grittner 2011-05-12 18:57:06 Re: Fw: [BUGS] BUG #6011: Some extra messages are output in the event log at PostgreSQL startup