Retry Service Best Practice in SonicESB Service

Posted by cwiseman on 30-Aug-2011 13:14

Maybe someone can suggest something what may be best here.

Got a process that receives messages and uses a SonicConnect Client service to consume an external web service.  If that external consumption fails, I will place the message on a queue to be handled in a retry fashion.  A process/service would 'watch' this retry queue and periodically wake up, drain the retry queue, and dispatch these messages back on the entry endpoint of the SonicConnect Client process/service which normally handles them.  I can handle retry maximums in the messages themselves, etc.

Is there a good pattern or best practice in designing the ESB service which will 'watch' the retry queue, wake up periodically, and work on the messages?  I don't want the retry to thrash but possibly wake and work on the queue messages, for example, every 10, to 30, to 60 seconds.

Thanks in advance.

All Replies

Posted by cwiseman on 02-Sep-2011 10:17

Anyone written a reliable ESB service which sleeps and wakes to process its inbox on a specified sleep interval?

I read somewhere that perhaps an activation daemon might possibly be used to control the service start/stop, is this a better approach?

Thanks.

Posted by sfritz on 02-Sep-2011 10:20

I am currently out of the office. I'll read your mail when I'm back on the 12th of September 2011. Should you have any matters that need attention prior to my return please contact customer support:

http://web.progress.com/support/contact-support.html

Your eMail will not be forwarded.

Stefan Fritz

Progress Software

Posted by kasch65 on 03-Sep-2011 04:35

For SonicConnect Services its propably the best way to let camel manage the retry behaviour. This preserves the message order and you don't have to configure any extra queues and services.

In the spring.xml File you can configure exception handling like this:

           
                org.apache.cxf.binding.soap.SoapFault
                org.apache.cxf.interceptor.Fault
               
           

When you define myDeliveryPolicy like this:

   
       
            value="${Adm4EsbService.Adm4EsbServiceHttpSoap11Endpoint.maximumRedeliveries}" />
       
            value="${Adm4EsbService.Adm4EsbServiceHttpSoap11Endpoint.redeliveryDelay}" />
       
   

you have what you normally want.

(Be careful, spring.xml is a generated file and overwritten on opload all)

Posted by mnair on 06-Sep-2011 00:06

Hi Chris,

You could use the fault handling feature - the Resubmit step - to handle the retries for you. With this approach you should add the service that implements the periodic polling functionality before the Resubmit step in the fault handler process

I have seen a delay service (backed by a queue), being used in similar use cases, owned by Progress PSO; this will require a PSO engagement.

In my personal opinion, implementing the polling/delay functionality in the service itself is much more flexible and manageable in the long run. Explicitly starting/stopping services from an external process will also work, but there is a larger window between the issue of the start/stop command and when the service actually responds to these lifecycle commands.You might also want to use the abort command instead of stop depending on your use case.

Thanks

-Mahesh

Posted by cwiseman on 08-Sep-2011 12:33

Mahesh,

Using a service which delays then reads a queue to resubmit the notification messages to the original sending service is where I wanted to go.  I felt as well I would have better management over the retries in this way.

What is the best architecture or design to have a service initialize on container startup, check a queue, process queue, sleep, wake again and process.  Service method is only traversed on messages to the entry endpoint, correct?  If it is a polling service, chances are those components will initialize in init or start methods.  I've created services in the past which create a Timer object in their init or start methods which wake to do work and sleep, but generally the service method was never used in real work processing in those services.  Should this type of service also not use the service method, but simply wake and attach to the retry queue to read and dispatch the notifications that need retry based on the business rules?

Thanks in advance.

Posted by jerem on 12-Sep-2011 10:14

Hello Chris,

You also use a custom scheduler service.

You can start a retry process in a cron fashion that handle message in queue and dispatch it to the right process or even to the right step in a process..

Regards

Chan Jeremy

Posted by cwiseman on 28-Oct-2011 09:59

Jeremy, I saw your other post about the Scheduler Service and it probably doing what I want.

I'll refer from here to this thread:  http://communities.progress.com/pcom/message/141759#141759

Posted by sedge on 08-Dec-2011 23:18

Hi Chris

How did you go with your retry process?

I've had the same requirement but I've used a Database to persist messages and have a custom service that does the re-submission. I use an algorithm that works on the basis that the longer an endpoint has been "down" the less frequently I should retry. I also use the principle that, once a message Type/endpoint is flagged as "failed" all new messages a queued behind the first failed message. At the retry I release the first message and, if that is successful, all the others get released. If not then the first failed message goes back to the top of the queue. That way I can maintain the delivery order as we have some applications that perfer to receive messages in the right sequence.

Happy to share some of the detals with you if you aren't too far down the track.

Regards

Steve

This thread is closed