GiST memory allocation

From: Neil Conway <neilc(at)samurai(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: GiST memory allocation
Date: 2004-11-01 14:32:45
Message-ID: 4186490D.2020506@samurai.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Memory allocation in access/gist/gist.c is pretty heinous, IMHO. There
are retail pallocs and pfrees all over the place, and the requirements
for which allocations need to be released and by whom is pretty messy.
AFAICS, GiST doesn't take any advantage of the palloc() infrastructure
beyond treating palloc() as a better malloc().

One relatively easy fix would be to create a per-backend "GiST context"
as a static variable in gist.c. At the external entry points to gist.c
(of which there are only three, gistbuild(), gistinsert(), and
gistbulkdelete()), the memory context would be initialized if necessary
and switched into. palloc() will still be used to allocate memory, but
most/all of the retail pfrees can be eliminated; instead, we can reset
the private memory context before returning to the caller of the GiST
entry point (the observation here is that 99% of the allocations done by
gist.c are for internal use only -- we rarely allocate anything that
needs to live longer than the current GiST API call). Additional resets
can be done where appropriate to reduce memory consumption -- for
example, we will definitely need a reset in gistbuildCallback().

I haven't looked that carefully at the various clients of the GiST API
(e.g. contrib/btree_gist and so on), but I think this should simplify
them as well: rather than needing to ensure they pfree all their
allocations, they can now assume that they are invoked in a relatively
short-lived memory context.

One downside is that the GiST API won't be reentrant, but I don't think
that's a major problem.

Comments? Is there a better way to clean up GiST's memory allocation?

(I looked primarily at gist.c; from a brief glance at gistscan.c, it
doesn't seem nearly so bad.)

-Neil

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2004-11-01 14:44:42 Re: psql and schemas
Previous Message Thomas Hallgren 2004-11-01 13:11:14 Re: charset/collation in values