I am using progress 10.2 B and trying to achieve multi threading using app server brokers. Initially I am creating around 70 connections to 4 brokers (distributed equally) and then running one persistent XYZ.p asynchronously on those previously created 70 connections.
Now I am running internal procedure from XYZ.p asynchronously using handle obtained while running XYZ.p as persistent. I this call I am passing internal procedure's name (EVENT-PROCEDURE) which is supposed to run after response received from asynchronous call. In that internal procedure I am marking server's as available.
What I observed is after some time and more load, EVENT-PROCEDURE is not getting executed, hence my main program is not able to understand agents availability and after some time, even if agents are seen as free on brokers, main program can not assign request to those agents as response from previous asynchronous call did not came or EVENT-PROCEDURE was not executed to mark agent as free.
This thing is making my multi threading architecture very much slow and each time I have to kill the process and restart it after some time. I guess that after execution of large number of request's at some point (either broker or main program) there seems resource contention.
Any one has came across same issue? Your observations and comments are much appreciated.
I've seen other issues with asynchronous calls where I needed to increase the -asyncqueuesize parameter. Perhaps you could try that? See documentation.progress.com/.../index.html
Also:
- What's the appserver's operating mode ?
- Are there any messages in the appserver's .broker.log that indicate a lack of available agents ?
@Frank: brokers operating mode is state-less and there are no error messages for lack of available agents. Even when I checked broker status, it's showing many agents as available. What I feel is after some load, event-procedure which is supposed to run after after asynchronous call gets completed, is not running , hence calling procedure is not able to mark that agent as free (logically marking it as free to avoid overloading broker).
I assume you have verified that the request is actually getting executed on the server and is thus done even though the PROCEDURE-COMPLETE event procedure is not running.
I would also recommend upping asyncqueusize
Hey Guys,
I tried increasing asyncqueusize parameter. But it is not increasing performance. I guess either my main thread (one which is assigning request to agents in asynchronous way) or agents (one which are processing request) getting slowed down after processing certain number of request. it slows down for some time like 60-90 sec then again it reached to initial speed. This cycle continues till all request are processed. I guess I am missing something like some resource or queue is not free so for 60-90 sec its slowing down then once that resource or queue is cleared, its getting good speed. Is there any other parameter which I need to change ?
I don't quite understand your model. Let me recap what I understand. I'm not sure I have #4 correct:
1. You have multiple AppServer connections.
2. You run a persistent procedure asynchronously on each connection.
3. You do another async RUN on each connection to run an internal procedure in the persistent procedure.
4. When the response comes back from the internal procedure, you know that connection is free so you send a new async RUN request?
#4 sounds like you're trying to manage the traffic on the connections yourself when the AVM already manages it for you. Here is what the doc says about asyncqueuesize:
As each client connects, request buffers are generated and the size of each buffer determined. Both send and receive requests can be queued on a client depending how the program is implemented. When the buffers are filled, additional queued requests are stored in a temporary table associated with that client session. If you set the value of -asyncqueuesize to 0, all requests are placed in the temporary table. Performance may vary based upon whether requests are stored in memory or in the temporary table. So you may want to choose this value based upon how much data you expect to flow between the client and the AppServer for asynchronous requests.
But if you're not sending new requests until the previous one is done, then asyncqueuesize is not going to affect you at all - unless perhaps you are getting large amounts of data back from each call this might improve performance on the receive side if the app is often busy and not frequently looking for events and thus not handling the data in a timely manner. Of course you do have to be waiting for events in order for your PROCEDURE-COMPLETE handlers to run.
@Laura: Your understanding is correct including #4. Yes I am waiting for response to come back and then only assigning new request. So as per info in doc for asyncqueuesize, its not going to affect performance in my model.
What I am not able to understand is I see good execution speed for some time say 10 min. then it slows down for next 1 min. Again in next 1-2 min it reaches to good execution speed.
Don't know. You should look at the AppServer log file and see if the slow down is on the client or the server (i.e., the time it actually takes to run the routine). You could also log on the client when you're PROCEDURE-COMPLETE routines run and compare to when the routine ended on the server.