Re: Pluggable Indexes

On Thu, 2009-01-22 at 00:29 +0300, Oleg Bartunov wrote:

> I'm rather keen on Pg extendability, which allowed me and Teodor to
> work on many extensions. Yes, first GiST we inherited from early
> academic research and was more like a toy. We still have several TODO
> items about GiST interface (incorporate SP-GiST).

Sounds good.

> I'm not sure about specific patch Simon advocate, but as soon as it
> doesnot introduces any threat to the whole database cluster health
> (for example, WAL spamming) I think we can apply it.

Currently you can write any crap you want to WAL from any plugin, as
long as it looks a lot like an existing WAL message type. If you crash
then we'll read that crap and (probably) crash again. That is already a
risk.

The rmgr plugin provides a way to handle user-defined WAL messages. The
patch is recovery-side only and is designed to complement the indexAM
APIs, which are normal-running-side only. Best way to think of it is as
another 5 functions on index access method interface that allow you to
implement recoverable index plugins. (Remembering that dynamic index
plugins are already allowed by Postgres).

So the patch does not provide any additional way of *writing* WAL, it
just provides a way of reading it and then taking action.

Rmgr plugins would allow you to simply ignore certain kinds of WAL,
apply data in a user defined manner or filter it etc.. So if you come
across a buggy index, you can turn off the WAL for that index type and
then recover the database without those indexes. Or dynamically patch
the code for that index type and recover. You'll get Postgres back up
faster with this patch than without it, in many cases.

> Other question, why don't improve GiST to allow support of more indexes ?
> bitmap indexes could be implemented usin g GiST.

I'm not advocating any particular type of index here, just the ability
to make index plugins robust. There is no other way of doing this, i.e.
it can't be done by an external module etc..

I'll avoid discussing index design with you :-)

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support