WIP: pre-upgrade page reservation

I attached contrib module which is base of preupgrade script. It should be part
of 8.4, but it will be required for 8.4->8.5 upgrade.

This part contains space reservation for heap/toast relations. The idea is that
relation is read and each block is checked if there is enough free space. Tuples
which will not be visible after upgrade are not count. If there is no space,
then simple_heap_update on tuple(s) until we release enough space.

BTree space reservation is more complicated. I plan to use _bt_split and split
page to two half pages with following code:

firstright = _bt_findsplitloc(rel, page, InvalidOffsetNumber, 0,&newitemonleft);
_bt_split(rel, buffer, firstright, InvalidOffsetNumber, 0, NULL,newitemonleft);
_bt_insert_parent(rel, buffer, rbuffer, stack, is_root, is_only);

Because both functions (_bt_findsplintloc, _bt_split) expect that we want to
insert new item, It will requires modification to accept InvalidOffsetNumber.

Another problem is to build stack which require to use deep tree scan. I hope
that it will not require exclusive lock on index.

I'm not yet look on hash, gist and gin. I think that hash index should be easy,
because index tuples can be moved into new bucket page. (Note: general problem
with hash index is still bitmap pages).

I guess solution for Gist index should be similar to BTree, but I don't have any
idea about GIN.

Comments, ideas, better solutions?

thanks Zdenek

PS: This patch requires previous patch which implemented space reservation
functionality.

Zdenek Kotala wrote:
> BTree space reservation is more complicated.

Do you need to pre-reserve the space for b-tree? I think you can just
split it at upgrade, in the new version. The problem with doing that for
heaps is that to move a heap tuple you need to update the index
pointers, but for indexes there's no such restriction.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas napsal(a):
> Zdenek Kotala wrote:
>> BTree space reservation is more complicated.
>
> Do you need to pre-reserve the space for b-tree? I think you can just
> split it at upgrade, in the new version. The problem with doing that for
> heaps is that to move a heap tuple you need to update the index
> pointers, but for indexes there's no such restriction.

The problem is that I need to know parent and modify parent as well. But you
don't know what is your parent node. You need to know root and go from root.
It is why I think that it is not doable online.

Correct me if I'm wrong.

thanks Zdenek

Zdenek Kotala wrote:
> Heikki Linnakangas napsal(a):
>> Zdenek Kotala wrote:
>>> BTree space reservation is more complicated.
>>
>> Do you need to pre-reserve the space for b-tree? I think you can just
>> split it at upgrade, in the new version. The problem with doing that
>> for heaps is that to move a heap tuple you need to update the index
>> pointers, but for indexes there's no such restriction.
>
> The problem is that I need to know parent and modify parent as well. But
> you don't know what is your parent node. You need to know root and go
> from root.
> It is why I think that it is not doable online.

Oh, you're planning to walk the B-tree in index order, not physical
order, so that you always have the stack for inserting the parents? You
don't necessarily need the stack, if you're not worried about
performance. _bt_insert_parent will scan the next level up to find the
parent in that case. That's slow, but so is walking the B-tree, and I'd
expect it to be rare that you need to split b-tree pages at upgrade anyway.

(I still think you're distracted, BTW. There's zero evidence that we'll
need any of this for the 8.4->8.5 upgrade. And if we do, we don't know
for sure that this will solve the problem, whatever the problem is.)

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas napsal(a):
> Zdenek Kotala wrote:
>> Heikki Linnakangas napsal(a):
>>> Zdenek Kotala wrote:
>>>> BTree space reservation is more complicated.
>>>
>>> Do you need to pre-reserve the space for b-tree? I think you can just
>>> split it at upgrade, in the new version. The problem with doing that
>>> for heaps is that to move a heap tuple you need to update the index
>>> pointers, but for indexes there's no such restriction.
>>
>> The problem is that I need to know parent and modify parent as well.
>> But you don't know what is your parent node. You need to know root and
>> go from root.
>> It is why I think that it is not doable online.
>
> Oh, you're planning to walk the B-tree in index order, not physical
> order, so that you always have the stack for inserting the parents?

Yes, it was a idea.

> You
> don't necessarily need the stack, if you're not worried about
> performance. _bt_insert_parent will scan the next level up to find the
> parent in that case. That's slow, but so is walking the B-tree, and I'd
> expect it to be rare that you need to split b-tree pages at upgrade anyway.

Cool. I overlooked it.

> (I still think you're distracted, BTW. There's zero evidence that we'll
> need any of this for the 8.4->8.5 upgrade. And if we do, we don't know
> for sure that this will solve the problem, whatever the problem is.)

We made a decision in a previous thread that we need space reservation when we
want to have CRC field in page header to prevent space expansion problem in page
conversion during upgrade. I think that currently we know what is necessary for
8.2->8.3/4 upgrade. What problems we can expect. We don't know if these kind of
changes happen in future or not. We know only about CRC at this moment. But I
supposed to prepare PostgreSQL to deal with all this issues. I'm not much happy
with idea to backport a lot of code to older version.

IIRC, we don't plan to backport space reservation back into 8.2, because ...
Why it should be accepted for 8.4 when 8.5 will be released?

Maybe I miss something or maybe I have lost in mailing thread and opinions ...

Zdenek