Following on from presentations in Boston and Finland, dot.r is pleased to announce the open source Stomp project, available immediately.
Download from either http://www.dotr.com or https://bitbucket.org/jmls/stomp , the dot.r stomp programs allow you to connect your progress session to any other application or service
that is connected to the same message broker.
Open source , free message brokers that support Stomp are :
Fuse (http://fusesource.com/products/fuse-mq-enterprise/) [a Progress company now owned by Red Hat inc]
====
Fuse MQ Enterprise is a standards-based, open source messaging platform that deploys with a very small footprint. The lack of license fees combined with high-performance, reliable messaging that can be
used with any development environment provides a solution that supports integration everywhere
ActiveMQ (http://activemq.apache.org/)
=======
Apache ActiveMQ (tm) is the most popular and powerful open source messaging and Integration Patterns server. Apache ActiveMQ is fast, supports many Cross Language Clients and
Protocols, comes with easy to use Enterprise Integration Patterns and many advanced features while fully supporting JMS 1.1 and J2EE 1.4.
Apache ActiveMQ is released under the Apache 2.0 License
RabbitMQ
=======
RabbitMQ is a message broker. The principal idea is pretty simple: it accepts and forwards messages. You can think about it as a post office: when you send mail to the post box you're pretty sure that Mr.
Postman will eventually deliver the mail to your recipient. Using this metaphor RabbitMQ is a post box, a post office and a postman.
The major difference between RabbitMQ and the post office is the fact that it doesn't deal with paper, instead it accepts, stores and forwards binary blobs of data - messages.
--------------------------------
Please feel free to log any issues on the https://bitbucket.org/jmls/stomp issue system, and fork the project in order to commit back all those new features that you are going to add ...
dot.r Stomp uses the permissive MIT licence (http://en.wikipedia.org/wiki/MIT_License)
Have fun, enjoy !
Julian
Hi Julian,
I had troubles installing ActiveMQ 5.6.0 on Windows 7 (64 bit).
You may want to also include this link "http://stackoverflow.com/questions/4626824/how-do-i-install-activemq-as-a-64-bit-service-on-windows" in your notes, it helped overcome this issue.
Cheers
David
P.S. Thanks for sharing... time to play!
I will do so, many thanks for the heads up.
Any issues, drop me a line.
Julian
Hi Julian,
Found a couple of issues and have some suggestions for improvement. Whats the best way to send that info to you?
Thanks
David
fantastic.
By far, the best way would be to fork, make your changes to your
repository, and then create a pull request for me to merge into the
main repository
have a look at https://confluence.atlassian.com/display/BITBUCKET/Forkingabitbucket+Repository
Thanks very much !
Julian
Done.
Be gentle, its my first fork! (haha sorry, coldn't help myself! )
David.
grin
Hi Julian,
This is interesting stuff indeed!
Some minor suggestions:
- Add a ROUTINE-LEVEL statement to StompClient.cls
- Add a bracket to the following example on the web page: dotr.Stomp.Helper.SendMessage:ToPersistentQueue("MyQueue","a test message".
Thank you for your contribution to the OpenEdge community!
-peter
jmls wrote:
I will do so, many thanks for the heads up.
Any issues, drop me a line.
Julian
Today I got the 'oooop' message:
IF SocketHandle:BYTES-WRITTEN NE lNumBytes THEN MESSAGE "oooop" VIEW-AS ALERT-BOX.
What does that indicate?
Heh. That means that for some reason, not all of the data was written
to the server. Very unusual. It can happen if the server crashes or
shuts down with active clients, or if the server can't process all the
data in one hit. What size chunks are you sending ? There are various
options in the ActiveMQ configuration to increase socket data size etc
I should write some error handling code round that to handle this .
I will at some stage
Julian
On 11 October 2012 13:03, Peter van Dam
jmls wrote:
Heh. That means that for some reason, not all of the data was written
to the server. Very unusual. It can happen if the server crashes or
shuts down with active clients, or if the server can't process all the
data in one hit. What size chunks are you sending ? There are various
options in the ActiveMQ configuration to increase socket data size etc
I should write some error handling code round that to handle this .
I will at some stage
Julian
On 11 October 2012 13:03, Peter van Dam
The chunks are small, and I am only receiving. But there were over 400 queued messages waiting on the server and for some reason they were not being processed by the client.
The server never faltered. When I closed the client because I noticed it was not processing messages I got the ooops message. I rely on GC when closing the client.
When I say closing the client I am not closing the OE session, just the menu program that also invokes the listener.
Hmm. That error is definitely in the sendframe method of StompConnection.cls
It could it be that the client connected , tried to subscribe to a queue and then lost the connection before it had a chance to send the subscribe frame. Or that it did connect , and you're sending receipt frames , and it got disconnected at some stage.
That would explain the server not sending the messages (the client is no longer connected) and the alert-box doesn't allow the client to be GC'd
try replacing the error message with this block of code and see what happens
if SocketHandle:bytes-written ne lNumBytes then
do:
case true:
when not valid-handle(SocketHandle) then assign err = "Socket handle invalid".
when not SocketHandle:connected() then assign err = "Socket is disconnected".
otherwise assign err = substitute("Unexpected write error on socket Bytes written [&1], expected to write [&2:]",SocketHandle:bytes-written, lNumBytes).
end case.
message err view-as alert-box warning.
end.
OK did that.
Now we wait
this is how the write method looks like in my own socketoutputbuffer... maybe can be easily adapted there
method override public void write( input bytes as memptr, input offset as integer, input len as integer ):
if bytes eq ? or offset eq ? or len eq ? then
undo, throw new NullPointerException().
if len eq 0 then
return.
if offset lt 0 or len lt 0 or offset + len gt get-size(bytes) then
undo, throw new ArrayIndexOutOfBoundsException().
checkClosed().
do while len gt 0:
outSocket:write(bytes, offset + 1, len) no-error.
if outSocket:bytes-written eq 0 then
undo, throw new IOException().
assign
offset = offset + outSocket:bytes-written
len = len - outSocket:bytes-written.
end.
end method.
thanks Marian
I will do that .
OK received the 'socket is disconnected' message today 3 times in a row after closing the menu.
I suppose socket disconnections can happen when there is a glitch on the network (this is actually running over the internet so no wonder).
However, what is strange is that I am getting these only after closing the menu and several times.
If it happened earlier I could try a (silent) reconnect before displaying an error message.
Any ideas?
Are you running on EC2 ?
We have had some major issues with the network "glitching" , so
clients like putty get disconnected after a very short time .
Stomp also has the problem, but so does ActiveMQ using jms clients, so
it's not progress
Also, which "menu" are you talking about ? Can you manually close the
client before you close the app ? GC does not always delete objects in
the "right" order, so sometimes we manually delete objects in the
order that we want them destroyed
On 12 October 2012 16:58, Peter van Dam
marianedu wrote:
this is how the write method looks like in my own socketoutputbuffer... maybe can be easily adapted there
method override public void write( input bytes as memptr, input offset as integer, input len as integer ):
if bytes eq ? or offset eq ? or len eq ? then
undo, throw new NullPointerException().
if len eq 0 then
return.
if offset lt 0 or len lt 0 or offset + len gt get-size(bytes) then
undo, throw new ArrayIndexOutOfBoundsException().
checkClosed().
do while len gt 0:
outSocket:write(bytes, offset + 1, len) no-error.
if outSocket:bytes-written eq 0 then
undo, throw new IOException().
assign
offset = offset + outSocket:bytes-written
len = len - outSocket:bytes-written.
end.
end method.
Error handling code similar to the above has been implemented and in the latest push. Thanks Marian.
thanks very much for the great comments.
Both of your suggestions are implemented in the latest version
Thanks for trying it out !
You're welcome Sir.
See you have more focus on that one, maybe something you can pick-up from the unfinished attempt I had on the subject http://sourceforge.net/p/progress-jms/code/102/tree/net.sourceforge.progress.jms/
I'm sure that I will be able to steal lots of ideas from it
I am but a "dwarf standing on the shoulders of giants" http://en.wikipedia.org/wiki/Standing_on_the_shoulders_of_giants
Are you running on EC2 ?
We have had some major issues with the network "glitching" , so
clients like putty get disconnected after a very short time .
Stomp also has the problem, but so does ActiveMQ using jms clients, so
it's not progress
Also, which "menu" are you talking about ? Can you manually close the
client before you close the app ? GC does not always delete objects in
the "right" order, so sometimes we manually delete objects in the
order that we want them destroyed
I am running over a normal internet connection (just for testing purposes).
I have noticed that every morning the socket connection seems to have been lost overnight. This means that I don't know which messages I missed.
The problem is that there is no 'disconnected' event I guess.
So I am thinking of reconnecting the client every 10 minutes or so to avoid this issues. Is that normal practice?
I am running over a normal internet connection (just for testing purposes).
I have noticed that every morning the socket connection seems to have been lost overnight. This means that I don't know which messages I missed.
The problem is that there is no 'disconnected' event I guess.
So I am thinking of reconnecting the client every 10 minutes or so to avoid this issues. Is that normal practice?
Is the server on the internet, or both client and server ?
If you use the latest version on bitbucket, I've added some error handling for socket disconnection. Stomp will now post an "error" message with the body of the message stating that the socket has been disconnected. You can now choose how to handle this (reconnect etc)
let me know if it helps.
If you are using ActiveMQ 5.7.0 or above, you could try adding the heartbeat option to the server, which "pings" the client every x milliseconds. This may help to maintain the connection as it is active.
If you need to know which messages that you have "missed" you can always use a queue for the one subscriber - the messages will build up until that consumer joins
Also, have a look at the Apache Apollo project (http://activemq.apache.org/apollo/documentation/user-manual.html) , which is going to the base of ActiveMQ 6 - there is an option to "retain" topic messages, which means that the last message sent to the topic is sent to each new subscriber
Peter, did you get a chance to look at the updates ?
We're running with ApacheMQ 5.8 snapshot, seems very sold pumping hundreds of thousands of messages through per day
Just thought I'd post some details of stomp in real world use
A snapshot of the ActiveMQ in operation at our call center.
The "queue" in "pbx,queue" is not a stomp queue, it's an inbound call queue
Each topic is automatically deleted after 30s inactivity
Name ↑ | Number Of Consumers | Messages Enqueued | Messages Dequeued |
---|---|---|---|
ActiveMQ.Advisory.Connection | 0 | 2515232 | 0 |
ActiveMQ.Advisory.Consumer.Queue.ActiveMQ.DLQ | 0 | 10 | 0 |
ActiveMQ.Advisory.Consumer.Topic.pbx.queue.info... | 0 | 1457 | 0 |
ActiveMQ.Advisory.Consumer.Topic.pbx.queue.warn... | 0 | 1457 | 0 |
ActiveMQ.Advisory.MessageDLQd.Topic.pbx.queue.i... | 0 | 13 | 0 |
ActiveMQ.Advisory.MessageDLQd.Topic.pbx.queue.i... | 0 | 5443 | 0 |
ActiveMQ.Advisory.MessageDLQd.Topic.pbx.queue.i... | 0 | 423 | 0 |
ActiveMQ.Advisory.MessageDLQd.Topic.pbx.queue.i... | 0 | 307 | 0 |
ActiveMQ.Advisory.MessageDLQd.Topic.pbx.queue.i... | 0 | 114 | 0 |
ActiveMQ.Advisory.MessageDLQd.Topic.pbx.queue.w... | 0 | 10 | 0 |
ActiveMQ.Advisory.MessageDLQd.Topic.pbx.queue.w... | 0 | 4642 | 0 |
ActiveMQ.Advisory.Queue | 0 | 1 | 0 |
ActiveMQ.Advisory.Topic | 0 | 15 | 0 |
debtnet.info.db.update.Campaign | 0 | 52 | 0 |
debtnet.info.db.update.CampaignData | 0 | 602618 | 0 |
debtnet.info.db.update.CampaignNumber | 0 | 541743 | 0 |
debtnet.info.db.update.ExtraData | 0 | 643 | 0 |
debtnet.info.db.update.Plan | 0 | 147 | 0 |
debtnet.info.db.update.Queue | 0 | 30 | 0 |
debtnet.info.db.update.SkillMatrix | 0 | 3 | 0 |
pbx.queue.info.agentconnected | 53 | 3357 | 14 |
pbx.queue.info.hangup | 53 | 4552 | 385 |
pbx.queue.info.newcall | 53 | 4700 | 309 |
pbx.queue.info.team.removed | 53 | 25 | 0 |
pbx.queue.info.tryagent | 53 | 50957 | 7929 |
pbx.queue.info.voicemail | 53 | 458 | 67 |
pbx.queue.warning.noagent | 53 | 47885 | 247530 |
pbx.queue.warning.timedout | 53 | 222 | 0 |
and the broker details
Name | localhost |
Version | 5.8-SNAPSHOT |
ID | ID:prometheus-1069-1350291226059-0:1 |
Uptime | 7 days 3 hours |
Store percent used | 0 |
Memory percent used | 1 |
Temp percent used | 0 |
Hi Julian,
I would like to use synchronous (request-response) messaging. Therefore support for properties such as 'correlation-id' and 'reply-to' should be added.
Currently this is not quite possible since although there is a StompMessage class, this class is not used for sending messages. The current header is constructed hardcoded in the Send() method in StompClient.cls as a string.
I would rather start using the StompMessage class so I can add those properties and be more flexible in the future.
As I am writing this I realize I suppose I could implement that without breaking the code by adding overloads.
What do you think?
-peter
Hi Peter - have a look at the WithHeader method on the stompclient. This method allows you to add headers of your own choice
Julian
jmls wrote:
Hi Peter - have a look at the WithHeader method on the stompclient. This method allows you to add headers of your own choice
Julian
OK I guess that will work.
rbf wrote:
jmls wrote:
Hi Peter - have a look at the WithHeader method on the stompclient. This method allows you to add headers of your own choice
Julian
OK I guess that will work.
I am trying to do this but it does not seem to work.
I think the headers are OK but there is this thing called creating a Temporary Queue that I don't seem to be able to achieve.
Do you know how to do that?
right, I see part of the problem
given this scenario
a) producer subscribes to /temp-queue/response
b) then sends a message with reply-to header set to "/temp-queue/response"
c) the consumer gets the message , with reply-to set to "/remote-temp-queue/ID\cjmls-study-49161-1353295140603-2\c463\c1"
d) and responds to that reply-to address
e) sure enough, message appears on the queue list of ActiveMQ , but with 0 consumers, so the message just sits there waiting
f) the producer is also just sitting around, waiting for the message as well, but is obviously not subscribed to the queue it needs to be
I seem to smell a bug somewhere. ActiveMQ is meant to automatically subscribe the producer to the /remote-temp-queue/ID\cjmls-study-49161-1353295140603-2\c463\c1 queue, but either it is not, or the subscription is terminated before the response.
I have asked the ActiveMQ folks for their read on the situation.
In the meantime, we don't use temporary queues
Each client that uses a stomp connection gets it's own guid using guid(generate-uuid) (let's call it "myguid") , and subscribes to that queue (/queue/myguid) . If we want a reqeust-response, set the reply-to to be "/queue/myguid", and manipulate the other headers as required.
The consumer of the message should use the reply-to header to send the information back to the orginal producer (in this case, using the /queue/myguid queue )
HTH
Julian
jmls wrote:
In the meantime, we don't use temporary queues
That is probably not an option, as the other party is and NMS client that conforms to the rules http://blog.cylewitruk.com/2011/08/synchronous-request-response-messaging-with-nms-and-activemq/
right, but if the other party simply responds to the queue specified
in the reply-to header, then won't it get back to your abl client ?
ActiveMQ will only change the reply-to queue name if you specify
/temp-queue/yourqueue instead of /yourqueue/. If you specify
"reply-to: /myguid" then the nms client reply should be sent down the
myguid subscription.
There are settings in the ActiveMQ configuration to automatically
delete queues and topics that are no longer in use, something that
also helps provide temp-queue functionality
In the meantime, I'll let you know what I hear back
On 1 December 2012 20:21, Peter van Dam
Hi Julian,
The only way I could get this working was by officially creating a temporary queue in the broker.
I added the code below in StompClient.cls to achieve this.
I am happy now since this works fine so you might consider adding this to your code base.
METHOD PUBLIC CHARACTER SubscribeToTempQueue():
RETURN SubscribeToTemp("queue").
END.
METHOD PUBLIC CHARACTER SubscribeToTempTopic():
RETURN SubscribeToTemp("topic").
END.
METHOD PUBLIC CHARACTER SubscribeToTemp (p_type AS CHAR):
DEFINE VARIABLE nfram# AS CHARACTER.
DEFINE VARIABLE TempQueue AS CHARACTER.
ASSIGN
TempQueue = GUID(GENERATE-UUID).
CREATE TTDestination.
ASSIGN
TTDestination.Destination = SUBSTITUTE("/temp-&1/ID:&2", p_type, TempQueue)
TTDestination.ID = GUID(GENERATE-UUID).
ASSIGN nfram# = SUBSTITUTE("SUBSCRIBE~nid:&1~ndestination:&2~nack:auto~n~n", TTDestination.ID,TTDestination.Destination).
StompConnection:sendFrame(nfram#).
RETURN TempQueue.
END METHOD.
Hi Julian,
After an extensive investigation EMEA (kudos to Rob Debbage!) figured out that this is an OpenEdge bug that only hits you when you set -cpinternal UTF-8 -cpstream UTF-8 which I was doing in my test environment but luckily not (yet) in the production environment.
The issue is logged as bug# OE00230773.
Yeah, I got a message from Psc informing me that someone had logged this . Kudos, again, to Psc for letting me know.
And, I have to say, well done to you for finding this !
* jmls is just really glad it's not my code
Now I hope that you're finding it useful !
jmls wrote:
Now I hope that you're finding it useful !
Definitely because now I know I can confidently put this code in production as long as we don't use UTF-8 which luckily is not the case.
I have noticed that the error had not occurred on the two pilot sites but could not explain that. Now I can.
Your code is excellent!
Keep up the good work!
-peter
your cheque is in the post ..
On 22 January 2013 21:20, Peter van Dam
Sorry for digging up an old post, but are there any examples with message selectors? also can the message selector be changed without starting a new session?
Thanks,
Paul.
never them myself, but from these docs (activemq.apache.org/.../stomp-manual.html) all I think you need to do is add a header with the appropriate selector before you subscribe
> I have noticed that every morning the socket connection seems to have been lost overnight.
I know that this is far too late to be of any use to you -- but I thought I would add a comment for others who might find this on a search. We ran into a similar problem -- our connection to ActiveMQ would regularly hang for no obvious reason. What we found was this:
1) There was a state-aware firewall between the client and the server. A connection would be opened between the two, but after about 90 minutes the firewall would notice that there had been no traffic, and would drop the connection. Neither end was notified.
2) The ABL STOMP client system is installed on Cent/OS 7 system with the default TCP/IP configuration. That means that after 2 hours the TCP/IP stack noticed the lack of traffic. It would try to contact the remote system 10 times, separated by 75 seconds. Since the firewall had closed the connection, these attempts failed and the TCP/IP stack closed the local connection. Our ABL Stomp client was so informed, and it shut down.
This problem was resolved (in a manner of speaking) by adding
heart-beat:0,60000
to the (hard-coded) connect frame. This tells the ActiveMQ system to send a heart-beat every minute, keeping the network connection alive.
I don't know what was happening on the ActiveMQ server at the same time -- but it never dropped the stale consumer from the queue. Heartbeats would probably resolve that issue too -- but that's a separate conversation.