[RFC] CREATE QUEUE (log-only table) for londiste/pgQ ccompatibility

Hallo postgresql and replication hackers

This mail is an additional RFC which proposes a simple way to extend the
new logical replication feature so it can cover most usages of
skytools/pgq/londiste

While the current work for BDR/LCR (bi-directional replication/logical
replication)
using WAL is theoretically enought to cover _replication_ offered by
Londiste it
falls short in one important way - there is currently no support for
pure queueing,
that is for "streams" of data which does not need to be stored in the
source database.

Fortunately there is a simple solution - do not store it in the source
database :)

The only thing needed for adding this is to have a table type which

a) generates a INSERT record in WAL

and

b) does not actually store the data in a local file

If implemented in userspace it would be a VIEW (or table) with a
before/instead
trigger which logs the inserted data and then cancels the insert.

I'm sure this thing could be implemented, but I leave the tech
discussion to those
who are currently deep in WAL generation/reconstruction .

If we implement logged only tables / queues we would not only enable a more
performant pgQ replacement for implementing full Londiste / skytools
functionality
but would also become a very strong player to be used as persistent
basis for
message queueing solutions like ActiveMQ, StorMQ, any Advanced Message
Queuing Protocol (AMQP) and so on.

comments ?

Hannu Krosing