Re: Clock sweep not caching enough B-Tree leaf pages?

On 04/17/2014 09:38 PM, Stephen Frost wrote:
> * Greg Stark (stark(at)mit(dot)edu) wrote:
>> On Thu, Apr 17, 2014 at 12:21 PM, Stephen Frost <sfrost(at)snowman(dot)net> wrote:
>>> Ehhh. No. If it's a hot page that we've been holding in *our* cache
>>> long enough, the kernel will happily evict it as 'cold' from *its*
>>> cache, leading to...
>>
>> This is a whole nother problem.
>>
>> It is worrisome that we could be benchmarking the page replacement
>> algorithm in Postgres and choose a page replacement algorithm that
>> chooses pages that performs well because it tends to evict pages that
>> are in the OS cache. And then one day (hopefully not too far off)
>> we'll fix the double buffering problem and end up with a strange
>> choice of page replacement algorithm.
>
> That's certainly possible but I don't see the double buffering problem
> going away any time particularly soon and, even if it does, it's likely
> to either a) mean we're just using the kernel's cache (eg: something w/
> mmap, etc), or b) will involve so many other changes that this will end
> up getting changed anyway. In any case, while I think we should
> document any such cache management system we employ as having this risk,
> I don't think we should worry about it terribly much.

Note that if we somehow come up with a page replacement algorithm that
tends to evict pages that are in the OS cache, we have effectively
solved the double buffering problem. When a page is cached in both
caches, evicting it from one of them eliminates the double buffering.
Granted, you might prefer to evict it from the OS cache instead, and
such an algorithm could be bad in other ways. But if a page replacement
algorithm happens avoid double buffering, that's a genuine merit for
that algorithm.

- Heikki