Re: Sorting writes during checkpoint

[Greg Smith <gsmith(at)gregsmith(dot)com>]

On Tue, 15 Apr 2008, ITAGAKI Takahiro wrote:

> 2x Quad core Xeon, 16GB RAM, 4x HDD (RAID-0)

What is the disk controller in this system?  I'm specifically curious 
about what write cache was involved, so I can get a better feel for the 
hardware your results came from.

I'm busy rebuilding my performance testing systems right now, once that's 
done I can review this on a few platforms.  One thing that jumped out at 
me just reading the code is this happening inside BufferSync:

buf_to_write = (BufAndTag *) palloc(NBuffers * sizeof(BufAndTag));

If shared_buffers(=NBuffers) is set to something big, this could give some 
memory churn.  And I think it's a bad idea to allocate something this 
large at checkpoint time, because what happens if that fails?  Really not 
the time you want to discover there's no RAM left.

Since you're always going to need this much memory for the system to 
operate, and the current model has the system running a checkpoint >50% of 
the time, the only thing that makes sense to me is to allocate it at 
server start time once and be done with it.  That should improve 
performance over the original patch as well.

BufAndTag is a relatively small structure (5 ints).  Let's call it 40 
bytes; even that's only a 0.5% overhead relative to the shared buffer 
allocation.  If we can speed checkpoints significantly with that much 
overhead it sounds like a good tradeoff to me.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD