Re: Bulk Insert tuning

Simon Riggs <simon(at)2ndquadrant(dot)com> writes:
> Following patch implements a simple mechanism to keep a buffer pinned
> while we are bulk loading.

This will fail to clean up nicely after a subtransaction abort, no?
(For that matter I don't think it's right even for a top-level abort.)
And I'm pretty sure it will trash your table entirely if someone
inserts into another relation while a bulk insert is happening.
(Not at all impossible, think of triggers for instance.)

From a code structural point of view, we are already well past the
number of distinct options that heap_insert ought to have. I was
thinking the other day that bulk inserts ought to use a ring-buffer
strategy to avoid having COPY IN trash the whole buffer arena, just
as we've taught COPY OUT not to. So maybe a better idea is to
generalize BufferAccessStrategy to be able to handle write as well
as read concerns; or have two versions of it, one for writing and one
for reading. In any case the point being to encapsulate all these
random little options in a struct, which could also carry along
state that needs to be saved across a series of inserts, such as
the last pinned buffer.

regards, tom lane