Re: Listen / Notify - what to do when the queue is full

Greg Sabino Mullane wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: RIPEMD160
>
>
>> We still need to decide what to do with queue full situations in
>> the proposed listen/notify implementation. I have a new version
>> of the patch to allow for a variable payload size. However, the
>> whole notification must fit into one page so the payload needs
>> to be less than 8K.
>
> That sounds fine to me, FWIW.
>

Agreed. Thank you for all your work.

>> 1) drop new notifications if the queue is full (silently or with rollback)
>
> I like this one best, but not with silence of course. While it's not the most
> polite thing to do, this is for a super extreme edge case. I'd rather just
> throw an exception if the queue is full rather than start messing with the

+1

--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

On Mon, Nov 16, 2009 at 2:35 PM, Andrew Chernow <ac(at)esilo(dot)com> wrote:
>
>>> 1) drop new notifications if the queue is full (silently or with
>>> rollback)
>>
>> I like this one best, but not with silence of course. While it's not the
>> most
>> polite thing to do, this is for a super extreme edge case. I'd rather just
>> throw an exception if the queue is full rather than start messing with the
>
> +1

So if you guys are going to insist on turning the notification
mechanism isn't a queueing mechanism I think it at least behooves you
to have it degrade gracefully into a notification mechanism and not
become entirely useless by dropping notification messages.

That is, if the queue overflows what you should do is drop the
payloads and condense all the messages for a given class into a single
notification for that class with "unknown payload". That way if a
cache which wants to invalidate specific objects gets a queue overflow
condition then at least it knows it should rescan the original data
and rebuild the cache and not just serve invalid data.

I still think you're on the wrong path entirely and will end up with a
mechanism which serves neither use case very well instead of two
separate mechanisms that are properly designed for the two use cases.

--
greg

>
> That is, if the queue overflows what you should do is drop the
> payloads and condense all the messages for a given class into a single
> notification for that class with "unknown payload". That way if a
> cache which wants to invalidate specific objects gets a queue overflow
> condition then at least it knows it should rescan the original data
> and rebuild the cache and not just serve invalid data.
>

That's far more complicated than throwing an error and it discards user payload
information. Let the error indicate a rescan is needed.

--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/