Re: Virtual tuple slots versus TOAST: big problem

On Sat, 2005-11-19 at 12:22 -0500, Tom Lane wrote:
> "A better fix" seems to require passing the TupleTableSlot, not just the
> bare tuple, down to the toaster --- else there is no way for the toaster
> to update the data structure that it's accidentally invalidating. This
> seems like it might be a good idea anyway on performance grounds: we
> could save one cycle of heap_deform_tuple and heap_formtuple in the case
> where toasting is needed, if the toaster is invoked on the tuple while
> it's still in virtual-slot format. The problem is that given the
> current structure, that means changing the APIs of heap_insert and
> heap_update, or else making near-duplicate versions that take a
> TupleTableSlot instead of a bare tuple. Neither of these things seem
> real attractive.

We changed the heap_insert API for 8,1 so would it be a problem to
change it again for 8.2

Other API changes sound likely, so seems less of a problem.

> If we wanted to avoid forming a physical tuple until
> the last moment we'd also need to change the APIs associated with
> triggers, ie make them work on Slots not tuples. This'd be even more
> invasive. It would likely be cleaner and more efficient in the long
> run, but there's a lot of code to touch, and breaking user-defined
> triggers doesn't seem palatable at all.

But if there are no triggers, then no problem.

Not sure I like this idea, but we could support both old and new trigger
APIs. If that was marked in the catalog, then we'd know when to change
slots into tuples.

What is the performance loss/gain by waiting?

On a different note, I'd like to consider how we can reduce the overhead
for a HeapTuple for when we do HashAgg and HashJoin. Currently the
overhead is substantial, with at least 12 bytes unused in most cases,
out of a total current overhead of > 20 bytes per tuple. Reducing that
overhead would increase the limit at which we start to do sorts.

Best Regards, Simon Riggs