Re: Concurrency testing

On Fri, Oct 09, 2009 at 08:34:55PM +0200, Markus Schiltknecht wrote:
> Hi,
>
> David Fetter wrote:
>> 1. Test situations which require more than one connection, possibly
>> to different clusters, for example in the cases of Hot Standby and
>> Streaming Replication. This further divides into event-based and
>> time-based tests. It's this situation I had in mind when I posted,
>> and it was inspired by bugs I've seen, the most recent being in
>> writeable CTEs.
>
> Hm.. not sure what you mean by time-base tests,

Kick off x.
Wait time t1
Kick off y.
Wait time t2.
etc., etc.

> but to me that distinction sounds like regression vs. performance
> testing. Or do you think of time-based regression tests? I
> certainly favor regression tests that are event-based, as I've run
> into too many false errors due to unexpected timings already.

Totally agree about regression tests. There might be other kinds of
test--stress- or performance tests, for example--that would make more
sense time-based.

>> 2. Test situations which require *many* connections in order to
>> find issues caused, in essence, by load.
>>
>> Tsung seems well-suited to the latter.
>
> Agreed. I would also note here, that Tsung seems especially well
> suited for performance testing of clustered solutions, as you very
> likely have to cluster the testing agents as well to put a decent
> load on the SUT.

I'd *love* to see PostgreSQL solutions that would outrun a single
decent Tsung box, but I suspect that's at least a year or two away.

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate