Re: CLOG contention

On Thu, Jan 5, 2012 at 4:04 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> I hypothesize that there are actually two kinds of latency spikes
> here.  Just taking a wild guess, I wonder if the *remaining* latency
> spikes are caused by the effect that you mentioned before: namely, the
> need to write an old CLOG page every time we advance onto a new one.
> I further speculate that the spikes are more severe on the unpatched
> code because this effect combines with the one I mentioned before: if
> there are more simultaneous I/O requests than there are buffers, a new
> I/O request has to wait for one of the I/Os already in progress to
> complete.  If the new I/O request that has to wait extra-long happens
> to be the one caused by XID advancement, then things get really ugly.
> If that hypothesis is correct, then it supports your previous belief
> that more than one fix is needed here... but it also means we can get
> a significant and I think quite worthwhile benefit just out of finding
> a reasonable way to add some more buffers.

Sounds reaonable.

Patch to remove clog contention caused by my dirty clog LRU.

The patch implements background WAL allocation also, with the
intention of being separately tested, if possible.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services