Hi,
I'm curious as to how the Aggregator pattern (http://www.enterpriseintegrationpatterns.com/Aggregator.html) plays in conjunction with an ESB process.
I can (fairly) easilly write a custom service to perform the appropriate aggregation but i'm concerned as to any impact this might have on an ESB process specifically because in the simple case there will be processes where messages never emerge from the aggregation step and as such the process would never 'complete'.
Also, in a more complex case where the Aggregation might be time based (ie output after 10 seconds even if you haven't received a message) a message might emerge from a service without being associated with a true 'process'.
So in summary, i'd like to be able to draw out an ESB process where one of the steps is (some sort of) an Aggregation - what is the most appropriate way of representing this.
Thanks in advance
Jem...
Your questions are indeed valid concerns, but are very general in nature.
You would need to make some decisions about how your aggregator works to make sure that no important messages get lost. For example, you might want to collect 10 messages and summarize them before sending them on. What happens if you don't ever get 10 messages? Perhaps you have to provide a timeout (send 10 messages or as many as you have collected in an hour).
If you go for a timeout solution, you might not have actually collected any messages. What should you do in that case? If your custom aggregator service produces a message (rather than forwarding one that has a itinerary), it will go to the Exit Endpoint of the Service and as such, is not directly associated with any "process". That said, the aggregator Exit Endpoint might be another process's Entry Endpoint...
In addition, you'll probably want to make sure that you are aggregating messages with the same itinerary. If you choose not to, which of the original itineraries should be used?
These are all fairly general questions, that can't really be answered without knowing the specific use-case.
Even though I think Aggregator pattern is a good fit for Long Running Transactions, I would still vote for it to be made available as a service in Sonic Workbench. It could be implemented with a few run-time parameters - number of messages to wait for, JMS header that will match the n-tuple.
I agree with Jamie about the some inherent things-to-consider of such a service depending on the specific use-case, but I think the most common one will be TTL, which could be set as part of the message or it could be set as part of the Endpoint definition where the Aggregator service/ESB process will listen.
You can implement the aggregator pattern in a number of ways with the ESB, probably the simplest two are using the split and join service that allows multiple endpoints to be invoked in parallel, or the other is to essentially perform sequential aggregation whereby each component that needs to be aggregated is invoked in sequence in the process and the results of each invocation are added to the message traversing the ESB.
To implement many of the combinations of the aggregator pattern would in fact require something akin to a lightweight BPEL service, so customers find it better to use the BPEL service. For short running transactions the BPEL service can be run in the transient mode this then is very close to the implementation that you are probably looking for.
An alternative implementation of an aggregator is to use an event based pattern, where the initial aggregation event, e.g. getCustomerInformation is sent to an event channel and any service that can fulfill this is able to reply with a response. The responses are sent to an aggregating cache that can be implemented in technologies such as database that can then be queried etc. This has the benefit of not defining the scope of the aggregation at development time, but is defined at runtime based on the services that are currently deployed.
David
We have need to aggregate a group of messages and create a message with contents from each message. If all the parts don't arrive with in 15 mins, it should proceed with messages collected so far.
I have implemented a BPEL process, which has a correlation set containing the key which uniquely identifies the message set. There is an ESB process which receives the message and calls the BPEL process to start aggregation. But as the message arrives to the ESB process, it creates a new process instance for each message even though they belong to the same correlation set. Hence is no aggregation.
Do we have a sample, where aggregation was implemented using a BPEL process.