Wait construct and ESB

Posted by apandit on 12-Aug-2011 17:21

Hello,

Typically service bus does not support wait construct or polling. I have a need to wait for the service to be ready before I can initiate download the file.

The process is

1. Send a message to download a file. This will return a token

2. Keep checking the status if the file is ready using the token.

When ready

3. Initiate file downlaod using this token

Is there a way to implenet this solution in ESB?

Thanks for your help in advance.

- Ashish

All Replies

Posted by sfritz on 16-Aug-2011 00:32

You can start a polling/checking thread in the XQServiceEx start() method (or XQService init()).

In the service method you can react on received messages e.g. request token.

Pass the token to the polling thread.

as soon as the download is possible , download and use XQDispatcher dispatch() to deliver the message.

You could also split this logic up into multiple service types and use a ESB process construct.

But this would include custom logic as well and I do not see much benefit in it.

Posted by cipdwak on 16-Aug-2011 00:35

Hallo,

Ik ben momenteel afwezig tot 2 September 2011. Voor dringende zaken kan je best contact opnemen met het secretariaat CIP/BIS per telefoon op het nummer 02/238.04.24

Met vriendelijke groet,

Koen De Waele

Posted by apandit on 16-Aug-2011 11:39

Thank you for the reply.

In this scenario, the file to be downloaded is 400 Mb. Will this solution still work?

Thanks,

Posted by sfritz on 17-Aug-2011 02:40

In theory this should not matter...

BUT processing a 400MB file is not really something that is typical for an ESB/SOA.

The idea of a SOA approach is more to be event driven. That means many small messages/events instead of a big batch of data.

If you really have to process this 400MB of data I would split it immediatly.

e.g. if the file contains records then generate one message per record while reading the file.

Use the dispatch api to immediatly dispatch the record after reading it and then continue to the next record.

I'd also try to check whether the soruce system can produce one file per "record" instead of one big file.

Sending the complete file (400MB) over the ESB is a no go.

You will run out of resources very soon and it is a bad design.

hth

Stefan

Posted by apandit on 17-Aug-2011 11:26

I absolutely agree with you and it will be designed to break down the file before sending it out.

The steps would be

1. Request for the file (every 3 months)

2. Poll the service until file is ready

3. Download the file (takes around 1 hour)

4. Unzip the file

5. Read one record at a time (expected to have around 3 milion records)

6. Send each record (less than 1 kb) through ESB to appropriate services.

I am just hoping that steps 2-5 can be done using ESB with help of Java helper class.

Thanks for all your replies!

- Ashish

This thread is closed