10MB message size limit

Posted by chetanparekh on 04-Nov-2010 03:27

We are using Sonic v7.6.2. We come to know from Sonic MQ documentation that maximum message size limit is 10MB.
Does it apply to Sonic ESB Process exposed as a WebService?
Does it apply to Sonic ESB Process which access external WebService?

All Replies

Posted by tsteinbo on 04-Nov-2010 04:50

This is a soft-limit and a default value only. It is imposed by the messaging broker. You can change this value or setting it to unlimited by configuring the broker's advanced property CONNECTION_TUNING_PARAMETERS.MAX_MSG_SIZE

Quoting from the "Configuration and Management Guide", section "Configuring SonicMQ Brokers"

CONNECTION_TUNING_PARAMETERS.MAX_MSG_SIZE
— Specifies the maximum acceptable message size by setting its value as
a non-negative integer value that specifies the allowed per-message
size limit in megabytes. The default value is 10 MB. Setting the value to zero (0) disables the feature.

And yes, it applies to any of the mentioned use cases since it will typically touch the broker.

Thomas

Posted by chetanparekh on 04-Nov-2010 05:47

Thanks Thomas!

Posted by chetanparekh on 08-Nov-2010 03:02

Is 10MB message size a recommended size?

How can we arrived at ideal max message size for our application?

Does Sonic has any capacity/performance model that tracks message size/frequency/volumes with message latency and computing resources like memory or CPU requirements for the ESB?

Posted by tsteinbo on 10-Nov-2010 01:59

10 MB is what we test with but we have used larger message sizes before. Personally, I have used up to 100 MB for a single message. The larger you go though the more careful you need to be with what you are trying to do with the message. E.g. if you want to execute an XSL transformation then we have to parse to DOM, execute the transform, get a new DOM and serialize this again. Since a DOM is 5 time the original message and we have 2 of them we allready look at 1 + 5 + 5 + 1 = 12 times the orinignal message size that you need in heap in the ESB container doing the transform.

In summary, while you can go larger the risk is you run out of memory.

Thomas

Posted by Bill Wood on 10-Nov-2010 02:49

As a comment, the issue with passing 10M messages in WebService calls is that you often find some clients will time out waiting for the response.  If you have a choice of size, and are using HTTP, you might have better luck optimizing for smaller sizes.

Posted by chetanparekh on 16-Nov-2010 02:39

Thanks Thomas! 12x is a very scary number.
Do you have any idea as how much memory it takes to validate and transform CSV based message (source and destination both formats are CSV)? Third party CSV library like OpenCVS, JavaCSV will be used for data validation and transformation.

This thread is closed