Quaranteed delivery from queue to DB in service

Posted by marko.lepikult@helmes.ee on 26-Mar-2010 03:19

Here is the situation ...
I have wrote custom service which writes incoming messages into database (not using Database Service).
Occasionally JDBC connection drops, messages are dequeued and sent to Outbox as FAULT messages.
I'd like to acheive situation when messages stay in entrypoint queue until database write has been commited.
At least stay in that queue when problem is DB connection, otherwise can go to RME.
So far I have figured out 2 solutions:
1. write custom service which connects to queue on start() and/or timer, reads message, writes to DB and ACK's message;
2. make in ESB proecss CBR which cheks Outbox message content. If return message is FAULT, then redirect input part of message back to service Entry (kind of endless loop).
Which solution of these two is best from ESB viewpoint or is there any other solution for this case.
TIA
Marko

All Replies

Posted by tsteinbo on 26-Mar-2010 04:46

Marko,

which Sonic verison are you using? if 7.6 then you can use the service life cycle interface and put the service into a stopped state while the db is unavailable. Once the db connection is up again you can start it. This will achieve what you need, e.g. messages queuing.

Thomas

Posted by marko.lepikult@helmes.ee on 26-Mar-2010 05:34

Thanks Thomas

I am using 7.6.2.

This solution is option, but I'd like to get closer to perfect solution

How could I avoid losing 1..N messages which are dequeued atuomatically when service() is invoked?

Since connection losses are rare and short I'd prefer not to stop service but try again after while.

Seems that these requirements force using a

- loopback enqueing through ESB process  of dequeued messages

or 

- creating JMS connection to Entry queue  sending messages back in service() after a little delay()

Marko

Posted by tsteinbo on 26-Mar-2010 05:51

Marko,

your solution will also work, but I guess we can also agree on disagreeing. E.g. I would argue that using the stop/abort of a service instance is favourable because you will not

  • loose message order
  • you don't have to manually maintain this "loop"
  • you don't have to artificially introduce a wait state (e.g. you don't want to resubmit from the fault/rme to the entry ep immediately).

With the life cycle manager calls in place you can still retry after a while and just start the service again once the db is up.

My 2 cents

Thomas

Posted by marko.lepikult@helmes.ee on 26-Mar-2010 08:44

Ok, I'am convinced 

I'll stop service.

Last question.

If I understand correctly, then I still loose a message.

Message witch invoked service() method and caused jdbc exception.

Or is the message kept in queue  when I execute stop() in the middle of service().

Marko

Posted by tsteinbo on 26-Mar-2010 09:15

To not "loose" a message (read as go to RME) you need to catch the exception and must leave the service() method gracefully.

You still need to decide if you want to use stop or abort. From the docs:

  • Stop the service — Stopping the service             terminates all of the service's listener threads and closes             the underlying JMS consumer. The listener threads that are             processing messages when the stop was called, wait until the             service() method returns. An attempt is made to             deliver all outstanding messages in the service's             outbox/faultbox/Reject endpoint and acknowledge the incoming             message. If the underlying JMS session is closed, message             delivery to the outbox/faultbox/Reject endpoint may fail. If  the             QoS is at least once, the incoming message             is not acknowledged and the system tries to redeliver the  message when the             service is started again. If QoS is exactly once,             the incoming transaction is not committed and the system  tries to                redeliver the message when the service is started again. Once a  service             shuts down successfully, it does not consume any further  messages             from its entry endpoint until the container is restarted or  the             service starts itself. If another service that is yet to be             stopped calls a stopped service using intra-container  messaging,             the message is delivered to the stopped service.
  • Abort the service — Aborting the service             terminates the service's listener threads and closes the             underlying JMS consumer. The listener threads that are  processing             messages when the abort was called wait until the             service() method returns. All messages in the             outbox, faultbox, or Rejected message endpoint as a result  of the             service execution are not delivered, and the incoming  message is             not acknowledged. Once a service shuts down successfully it  does             not consume any further messages from its entry endpoint  until             the container is restarted. If another service that is yet  to be             aborted calls an aborted service using intra-container  messaging,             the aborted service does not process such requests. Aborting  a             service uses the same state transitions as that of a service             stop. The only difference is that in the case of abort a special             flag is set in the Service Lifecycle Manager, so that it is             possible to distinguish between a stop and an abort. You can also check                whether a service is aborting by calling the isServiceAborting() method.

Posted by marko.lepikult@helmes.ee on 26-Mar-2010 16:31

Thank you Thomas.

This solution is perfect. I'll use abort.

Marko

This thread is closed