Re: our buffer replacement strategy is kind of lame

On Sun, Aug 14, 2011 at 1:44 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> The big problem with this idea is that it pretty much requires that
> the work you mentioned in one of your other emails - separating the
> background writer and checkpoint machinery into two separate processes
> - to happen first.  So I'm probably going to have to code that up to
> see whether this works.  If you're planning to post that patch soon
> I'll wait for it.  Otherwise, I'm going to have to cobble together
> something that is at least good enough for testing.

No, the big problem with the idea is that regrettably it is just an
idea on your part and has no basis in external theory or measurement.
I would not object to you investigating such a path and I think you
are someone that could invent something new and original there, but it
seems much less likely to be fruitful, or at least would require
significant experimental results to demonstrate an improvement in a
wide range of use cases to the rest of the hackers.

As to you not being able to work on your idea until I've split
bgwriter/checkpoint, that's completely unnecessary, and you know it. A
single ifdef is sufficient there, if at all.

The path I was working on (as shown in the earlier patch) was to apply
some corrections to the existing algorithm to reduce its worst case
behaviour. That's something I've seen mention of people doing for
RedHat kernels.

Overall, I think a minor modification is the appropriate path. If
Linux or other OS already use ClockPro then we already benefit from
it. It seems silly to track blocks that recently left shared buffers
when they are very likely still actually in memory in the filesystem
cache.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services