Re: Listen / Notify rewrite

On Thu, Nov 12, 2009 at 3:09 PM, Andrew Chernow <ac(at)esilo(dot)com> wrote:
>>
>> What advantage is there in limiting it to a tiny size? This is a
>> 'payload' after all...an arbitrary data block. Looking at the patch I
>> noticed the payload structure (AsyncQueueEntry) is fixed length and
>> designed to lay into QUEUE_PAGESIZE (set to) BLCKSZ sized pages.
>>
>
> Hmmmm. Looks like the limitation comes from slru. The true payload limit
> is (8K - struct members) the way this is implemented.

So I'm beginning to think that adding a "payload" to NOTIFY is a bad
idea altogether.

The original use case NOTIFY was designed to handle was to implement
condition variables. You have clients wait on some data structure and
whenever someone changes that data structure they notify all waiting
clients to reread the data structure and check if their condition is
true. The condition might be that the data structure doesn't match
their locally cached copy and they have to rebuild their cache, it
might be that some work queue is non-empty, etc.

So the way to build a queue would be to use a table to hold your work
queue and then use NOTIFY whenever you insert into the queue and any
idle workers would know to go dequeue something from the queue.

It sounds like what people are trying to do is use NOTIFY as a queue
directly. The problem with this is that it was specifically not
designed to do this. As evidenced by the fact that it can't store
arbitrary data structures (nothing else in postgres is limited to only
handling text data, this would be an annoying arbitrary limitation),
can't handle even moderately large data efficiently, etc.

I'm beginning to think the right solution is to reject the idea of
adding a payload to the NOTIFY mechanism and instead provide a queue
contrib module which provides the interface people want.

--
greg