Re: GiST memory allocation

On Tue, 2004-11-02 at 02:20, Tom Lane wrote:
> Neil Conway <neilc(at)samurai(dot)com> writes:
> > (the observation here is that 99% of the allocations done by
> > gist.c are for internal use only -- we rarely allocate anything that
> > needs to live longer than the current GiST API call).
>
> You sure about that? In btree quite a lot of the allocs need to survive
> across the current scan.

I'm specifically talking about gist.c, which just handles index
creation, tuple insertion and tuple deletion -- AFAICS we only rarely
need to allocate anything that lives beyond the current API call. Scans
are handled by gistscan.c. It might be nice to make better use of memory
contexts in both, but it's gist.c that is particularly in need at the
moment, I think.

> I think this is not acceptable. It certainly wouldn't be acceptable for
> btree --- you couldn't even use a PL-language function as an index
> operator, because the PL itself would need to do system catalog accesses
> that could result in re-entrant btree scans. If you got away with it
> for GiST it would only be because GiST is a stepchild that the system
> doesn't depend on. That doesn't sound like the direction to go in.

One alternative is to create memory contexts for each insertion /
creation / deletion operation, but that is pretty ugly, and probably
inefficient for insertion/deletion.

I can't really think of a better solution than a static memory context.
I don't think the reentrency will be a problem right now, but if it
becomes a problem in the future we can solve it via some book-keeping
(e.g. on entry to GiST bump a counter, when the counter == 0 and we're
exiting a GiST API then reset the memory context).

-Neil