Re: Group Commit

On 11/14/2011 03:43 PM, Josh Berkus wrote:
>
>> Purpose is to provide consistent WAL writes, even when WALInsertLock
>> contended. Currently no "off" option, thinking is that the overhead of
>> doing this is relatively low and so it can be "always on" - exactly as
>> it is for sync rep.
>>
> Hmmm, have you had a chance to do any performance tests?
>

I was planning to run some later this week, but someone else is welcome
to take a shot at it. The inspiration for this change was the
performance scaling tests I did for sync rep last month. Don't recall
if I shared those with this list yet; I've attached the fun graph. Over
a slow international link with 100ms ping times, I was only getting the
expected 10 TPS doing sync rep with a single client. But as more
clients were added, so that a chunk of them were acknowledged in each
commit reply, the total throughput among all of them scaled near
linearly. With 300 clients, that managed to hit a crazy 2000 TPS.

The best scenario to show this patch working would be a laptop drive
spinning at a slow speed (5400 or 4200 RPM) so that individual local
commits are slow. That won't be 100ms slow, but close to 10ms is easy
to see. When adding clients to a system with a slow local commit, what
I've observed is that the scaling levels off between 750 and 1000 TPS,
no matter how many clients are involved. The hope is that this
alternate implementation will give the higher scaling in the face of
slow commits that is seen on sync rep.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us