Re: Placing hints in line pointers

On Mon, Jun 10, 2013 at 3:43 AM, Jeff Davis <pgsql(at)j-davis(dot)com> wrote:
>> We wouldn't need to do a FPW when a hint changes, we would only need
>> to take a copy of the ItemId array, which is much smaller. And it
>> could be protected by its own checksum.
>
> I like the direction of this idea.

One of the previous proposals was to move all the hint bits to a
dedicated area. This had a few problems:

1) Three areas would have meant we wold have needed some tricky
ability to relocate the third area since the current "grow from both
ends" technique doesn't scale to three. Throwing them in with the line
pointers might nicely solve this.

2) We kept finding more hint bits lying around. There are hint bits in
the heap page header, there are hint bits in indexes, and I thought we
might have found more hint bits in the tuple headers than the 4 you
note. I'm not sure this is still true incidentally, I think the
PD_ALLVISIBLE flag might no longer be a hint bit? Was that the only
one in the page header? Are the index hint bits also relocatable to
the line pointers in the index pages?

3) The reduction in the checksum coverage. Personally I thought this
was a red herring -- they're hint bits, they're whole raison d'etre is
to be a performance optimization. But if you toss the line pointers in
with them then I see a problem. Protecting the line pointers is
critically important. A flipped bit in a line pointer could cause all
kinds of weirdness. And I don't think it would be easy to protect them
with their own checksum. You would have the same problems you
currently have of a process updating the hint bit behind your back
while calculating the checksum or after your last WAL record but
before the block is flushed.

Now if this is combined with the other idea -- masking out *just* the
hint bits from the checksum I wonder if moving them to the line
pointers doesn't make that more feasible. Since they would be in a
consistent location for every line pointer, instead of having to check
on each iteration if we're looking at the beginning of a tuple, and
the only thing we would be counting on being correct before checking
the checksum would be the number of line pointers (rather than every
line pointer offset).

--
greg