Using a queue as a stack

Posted by yuruki on 22-Jun-2011 07:22

I have been looking for a way to store semi-persistent data in Sonic. Data will be fully updated (overwritten) periodically and read by many readers in their own pace. Reading and writing should never block and the readers should always get the latest version of the data. I know this should probably be done with an external database, but I was wondering if it's possible to take advantage of the SonicMQ queues (in a CAA environment, the data would be nicely replicated, too).

So, what I'm really after is a stack (LIFO), with some sort of cleanup to prevent it from growing continuously (I'm only interested in what's at the top of the stack). To achieve this we take a queue. Writers write to the queue normally, but reading uses QueueBrowser (in a custom service) to skip to the last message in the queue. This is all it takes to get the latest data from a queue, but the queue will keep on growing.

I suppose we can keep the queue size in check by setting the Time to Live value in the messages to a suitable level. This however means that in case the messages stop coming for some reason, even the latest data will eventually disappear from the queue. Queue should always keep at least one message, so this approach doesn't work. Second option involves setting message properties (JMSExpiration) with the help of QueueBrowser on all messages in the queue except the last. But setting properties (and headers) doesn't seem to work when messages are in the queue. Is it really not possible?

All Replies

Posted by sfritz on 22-Jun-2011 07:26

I'd go for a database but anyway....

You already have a queue browser.

Use this to track the JMSmessageIDs of all messages.

Use a receiver with a filter to consumer (=delete) all but the latest one.

hth

Posted by yuruki on 23-Jun-2011 00:24

Thanks, that would work. I take it that editing messages inside the queue is not allowed. I was hoping to set the Time to Live on all messages except the last. That way the service wouldn't have to remove the messages and there would be no danger of another reader deleting "the last message" of another reader that started a moment before a message was added.

This thread is closed