Background:
We have around 120 ESB containers in production and the number is still growing with each new deployment. The reason for this is; with every new process we are creating a new container.
For most of the new requirement, we are creating a new container of one or two process with two or three steps. The steps can include XML transform, file drop, CBR and custom services (not complex).
95 % of the containers in our production are having just 3 or 4 steps with max of 2 processes in it. The services in these containers are the above mentioned services only.
Problem due to this approach:
1. Its becoming difficult to manage the large number of containers.
2. For every new container a new connection is created. So our connections are also growing with containers.
Question 1: what are the factors which decide when to create a new ESB container? Is there a max limit to the number of services or process that should be allowed in a container? Does it create any performance impact if multiple processes are placed inside a container?
My understanding about the ESB container; It is a component which provides an environment for services to run and provides out-of-box functionalities such as life cycle management, security, etc. We can place as many related process and services in container to keep the project modular.
Please correct me if I have written something wrong or I have missed something.
Question 2: Is it a good practice to assign a separate connection to each container or can we share a single connection among multiple containers? There could be low and high traffic connection in my knowledge. Is it really needed that each container should be assigned a different connection? What factors decide when to create a new connection?
Hope I was able to convey my question.
Thanks for patiently reading this post ;).Please suggest.
-Habin
Do you follow any governance process?
Q1: More or less your understanding is correct. Normally people decide which processes and services to host in a ESB container by project or by deployment location. Sometimes it makes sense to have just one service (e.g. adapter/Connector service) on a specific server. In other situations it is depending on how to distribute the load. If you want to scale and you know your bottlenecks it might make sense to distribute those (same) services to multiple ESB containers on multiple machines. So you see it depends a lot.
There is no real limit on the number of services, processes etc.. Available memory will be a factor. In addition the Sonic Management Console will react slower if it has to deal with huge numbers of artifacts (to be improved in 8.1).
Depending on what you do on the ESB it might also be interesting to understand when the messages go to/from the JMS layer to the ESB. Is Intra Container Messaging (ICM) used or not etc.
We call this ICM chains. ESB process steps that are executed in a row in the same ESB container with ICM enabled are called a ICM chain. These steps are then executed in memory and do not touch the JMS layer. This approach has pros and cons compared to using JMS between every step. The ESB endpoint quality of service has also an impact on the behavior.
This is too much to cover it here in a forum post.
Q2: Again: "It depends" In theory I see no problem with sharing the connection. But it might sense to have multiple connections (maybe not really for every container) if it makes sense in the architecture. I'm a fan of keeping things simple and only introduce complexity of the performance/architecture/governance etc. requires it. If you go back to one (or a few) connections you will have to increase the sessions on the ESB connection. I'd recommend to do a load test before and after the changes to be able to tune the system accordingly.
Regards
Stefan
Thanks a lot.