Re: [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins

On 2014-04-09 18:13:29 +0530, Pavan Deolasee wrote:
> On Wed, Apr 9, 2014 at 6:02 PM, Andres Freund <andres(at)2ndquadrant(dot)com>wrote:
> > I've tried to reproduce problems around this (when I wrote this), but
> > it's really hard to construct cases that need more than 8 pins. I've
> > tested performance for those cases by simply not using the array, and
> > while the performance suffers a bit, it's not that bad.

> AFAIR this was suggested before and got rejected because constructing that
> worst case and proving that the approach does not perform too badly was a
> challenge. Having said that, I agree its time to avoid that memory
> allocation, especially with large number of backends running with large
> shared buffers.

Well, I've tested the worst case by making *all* pins go through the
hash table. And it didn't regress too badly, although it *was* visible
in the profile.
I've searched the archive and to my knowledge nobody has actually sent a
patch implementing this sort of schemes for pins, although there's been
talk about various ways to solve this.

> An orthogonal issue I noted is that we never check for overflow in the ref
> count itself. While I understand overflowing int32 counter will take a
> large number of pins on the same buffer, it can still happen in the worst
> case, no ? Or is there a theoretical limit on the number of pins on the
> same buffer by a single backend ?

I think we'll die much earlier, because the resource owner array keeping
track of buffer pins will be larger than 1GB.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services