SonicMQ/ESB Performance Tuning On Simple HTTP WS Exposed Pro

Posted by cwiseman on 03-Nov-2010 11:27

Can someone help out with identifying or recommending where some performance tuning can be performed on a simple process initiated by a client hitting an acceptor?

We are using the out of the box Mgmt Broker and dev_ESBTest to position a WS-Protocol handler in the HTTP Direct acceptor on the broker.  The acceptor receives a SOAP request and places the request on a topic destination which is the entry endpoint of a simple process.  The process contains the following components (3) in line:

  • An initial Xform service to strip the SOAP tags and leave the resulting body payload in message part 0
  • A custom service which
    • Calls a custom API passing part 0 XML to perform a database query
    • Query response XML replaces part 0 XML to return to client on outgoing envelope
  • A final Xform service to wrap SOAP tags around the response body payload in message part 0 (it is simple now, not trying to preseve initial soap headers, etc.)

Our custom service is positioned to perform the work of an operation of a web service with a distinct SOAPAction.  Ultimately, the SOAPAction will be used on entry to route to various operations but for now, it is front ended like this:

  • Broker acceptor contains custom WS-Protocol element with a mapping extension URL to entry endpoint topic of above mentioned process
  • The custom WS-Protocol was simply added to the http.DIRECT acceptor in the Mgmt Broker and set to listen on port 2580
  • Accessing http://host:1580/extensionURL?wsdl properly returns the WSDL from the SonicFS with the re-written port information - fine.

Logging in the custom service around the DB query/obtain response yields performance numers querying and getting a response from the database in the order of about .4 to .6 seconds.  Highly acceptable for our transaction and requirements and about what we've come to expect with all the optimzations we have pushed into this underlying service layer.  The overall round trip as metered by our test client is in the order of 2 to 5 seconds to consume the service on the acceptor and get a response.  We really expect and need this entire operation from hitting the acceptor to getting the response to the client in the order of 1 second or less - not adding much platform overhead to the database transaction.  We are porting these services from a BizTalk .NET implementation and obviously expect better performance from SonicMQ/ESB.  We've gotten this down to near or just below 1s on average using BizTalk with all of its SQLServer transacted management overhead on messages just to get the XML payload of the request into the orchestration.  We believe an overall response at or less than 1s is optimal having examined our custom services and layers with multiple profilers.

Some other items of note for completeness to provide insight into what we've modified or tweaked on the environment:

  • We have added the following system property to the dev_ESBTest MF container environment:  -Dorg.apache.commons.logging.LogFactory=org.apache.commons.logging.impl.LogFactoryImpl
    This allowed us to use our own, already built logging services based on log4j.  Our own jar which contains our log4j.xml file was added to the pre-pended classpath for the MF container to find our log4j.xml config file.
  • We have modified some of the Mgmt Broker buffer size defaults and increased them by about 4-5x as suggested by the SonicMQ Tuning and Performance Guide.  Our input message is approximately 325Kb, the response is generally smaller.
  • Intra-Container Messaging is enabled on the dev_ESBTest container so we don't go back and forth to the broker for the independent process steps.

How can we reduce or meter the SonicMQ/ESB overhead around the .4 to .6 seconds for the db transaction in the custom service?  What else can we look at?  Thanks in advance.

All Replies

Posted by cwiseman on 03-Nov-2010 11:47

Forgot to mention:

  • Sonic V8.0
  • WindowsXP SP3 Development Platforms
  • Client requester on one machine
  • SonicIWB with Broker Acceptor and dev_ESBTest  as detailed above and SQLServer DB on one machine

Posted by mnair on 03-Nov-2010 12:06

Hi Chris,

Do not use the dev containers in a performance testing scenario. Caching (of ESB configuration artifacts, supporting artifacts like stylesheets, etc.) is turned off by default in the dev container and as a result there is a performance penalty you have to pay. If you turn off the TEST_CONTAINER flag or if you deploy in to another container then I would expect to see a significant improvement.

Its a good idea to enable intracontainer messaging.

Since you are on ESB 8.0, have you considered using Sonic Connect? This will take care of the SOAP stripping and addition automatically and you can avoid the two XForm steps.

Thanks

-Mahesh

Posted by cwiseman on 03-Nov-2010 12:34

Mahesh, thanks for the insights.

My past experience with SonicMQ/ESB was pre-v8.0 and I wondered if the new SonicConnect inclusions would assist with the SOAP stripping and wrapping.  That aspect has always been a chore with implementing SOAP web services using the standard acceptors and handling operations in processes after branching.  I simply hadn't had the opportunity to look at implementing SonicConnect yet and wanted to get something quick and dirty to test with for our initial porting of the .NET/BizTalk Server application.  I will review this - I'm hoping also that handling the SOAPActions and SOAP headers is put into conveniences in Sonic Connect too.  We shall see what it entails.

I will post again with the initial results of removing the TEST_CONTAINER flag on the development ESB Container.

Posted by cwiseman on 03-Nov-2010 13:39

Setting TEST_CONTAINER_MODE to false improved initial performance significantly.  We got numbers much closer to what we would expect before trying further optimizations - such as you mention by using SonicConnect.

Thanks!

This thread is closed