PASOE only running two HTTP session requests concurrently

Posted by dbeavon on 04-Nov-2018 02:32

Does anyone know why my PASOE (using HTTP sessions) would only be allowing two session requests to be processed concurrently?

Here are the sessions and agents, as reported by the oemanager webapp

There are five sessions that have simultaneous requests from a .Net open client.  Only two are able to be processed at a time.

This only happens on a PASOE server with the production license, but not on my PDSOE workstation.  On my workstation all five can be processed at once without any blocking.  Both run 11.7.4.  My PDSOE workstation runs windows 10, and the PASOE server runs windows 2012 r2.

I don't configure session manager restrictions to be different than the default.

 
 maxABLSessionsPerAgent=200
 maxAgents=10
 maxConnectionsPerAgent=16

I'm not sure where the limit of two would be coming from.  Any help would be appreciated.

Thanks David

Posted by dbeavon on 05-Nov-2018 15:27

I finally figured out why a .Net open client would only allow two appserver requests to be sent to PASOE at a time.

By breaking the debugger, I noticed that most of my waiting callstacks looked like this.

 	mscorlib.dll!System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle waitableSafeHandle, long millisecondsTimeout, bool hasThreadAffinity, bool exitContext)
 	mscorlib.dll!System.Threading.WaitHandle.WaitOne(int millisecondsTimeout, bool exitContext)	
 	System.dll!System.Net.LazyAsyncResult.WaitForCompletion(bool snap)	
 	System.dll!System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest request, bool forcedsubmit)	
 	System.dll!System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest request, string connName)	
 	System.dll!System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint servicePoint)	
 	System.dll!System.Net.HttpWebRequest.GetRequestStream(out System.Net.TransportContext context)	
 	System.dll!System.Net.HttpWebRequest.GetRequestStream()	
 	Progress.o4glrt.dll!Progress.UBroker.Util.HTTPConnection.Post(byte[] data, Progress.Common.Util.NVPair[] headers, Progress.UBroker.Util.UBWebRequest webRequest)	


 

After a bit of googling,  I discovered that there is a "service point manager" which deliberately throttles the client to only send out a couple of requests at a time (to the same endpoint).  The related class that serves this purpose is System.Net.ServicePointManager.

https://stackoverflow.com/questions/1361771/max-number-of-concurrent-httpwebrequests


https://docs.microsoft.com/en-us/dotnet/api/system.net.servicepointmanager.defaultconnectionlimit?view=netframework-4.7.2

There is a fix to change the throttling. Here is some code that can change it....

        /// <summary>
        /// Static values used for HTTP clients.
        /// </summary>
        public static void SetStaticServicePointManagerProperties(
            int p_DefaultConnectionLimit = 100,
            bool p_UseNagleAlgorithm = false,
            bool p_UnlimitedServicePoints = true,
            int p_MaxServicePoints = 1000)
        {
            // Default limit should be at least 10
            System.Net.ServicePointManager.DefaultConnectionLimit = p_DefaultConnectionLimit;


            // Whether to use nagle (delayed packet transmission)
            System.Net.ServicePointManager.UseNagleAlgorithm = p_UseNagleAlgorithm;


            // Optional limit on service points
            if (p_UnlimitedServicePoints)
            {
                // 0 means there is no limit to the number of System.Net.ServicePoint objects.
                System.Net.ServicePointManager.MaxServicePoints = 0;
            }
            else
            {
                // Specify the maximum
                System.Net.ServicePointManager.MaxServicePoints = p_MaxServicePoints;
            }
        }

Hope this helps anyone who may encounter the same problem.  At this point I am doubtful that the Progress open client API directly exposes the ServicePointManager's  "DefaultConnectionLimit".  (especially since that API was retrofitted quite a bit after PASOE came along).   

Keep in mind that the change affects all HTTP requests from the same .Net app, not just PASOE.

All Replies

Posted by dbeavon on 04-Nov-2018 05:30

I'm seeing a few behaviors that are different when deploying to the remote PASOE production instance.  I will try to do some configuration comparisons, but I have a feeling it isn't my fault.  I found a KB that indicates there are some types of failures that specifically happen when PASOE isn't running on the same machine as the .Net open client:

"...the call will fail if the PASOE instance is not on the same machine" See

knowledgebase.progress.com/.../invalid-state-for-read-net-open-client-to-pasoe

Can anyone help me understand why PASOE would behave substantially differently when running it locally than when running on a remote machine?  I don't think it is simply the difference between the dev and prod license.

Thanks, David

Posted by dbeavon on 05-Nov-2018 15:27

I finally figured out why a .Net open client would only allow two appserver requests to be sent to PASOE at a time.

By breaking the debugger, I noticed that most of my waiting callstacks looked like this.

 	mscorlib.dll!System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle waitableSafeHandle, long millisecondsTimeout, bool hasThreadAffinity, bool exitContext)
 	mscorlib.dll!System.Threading.WaitHandle.WaitOne(int millisecondsTimeout, bool exitContext)	
 	System.dll!System.Net.LazyAsyncResult.WaitForCompletion(bool snap)	
 	System.dll!System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest request, bool forcedsubmit)	
 	System.dll!System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest request, string connName)	
 	System.dll!System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint servicePoint)	
 	System.dll!System.Net.HttpWebRequest.GetRequestStream(out System.Net.TransportContext context)	
 	System.dll!System.Net.HttpWebRequest.GetRequestStream()	
 	Progress.o4glrt.dll!Progress.UBroker.Util.HTTPConnection.Post(byte[] data, Progress.Common.Util.NVPair[] headers, Progress.UBroker.Util.UBWebRequest webRequest)	


 

After a bit of googling,  I discovered that there is a "service point manager" which deliberately throttles the client to only send out a couple of requests at a time (to the same endpoint).  The related class that serves this purpose is System.Net.ServicePointManager.

https://stackoverflow.com/questions/1361771/max-number-of-concurrent-httpwebrequests


https://docs.microsoft.com/en-us/dotnet/api/system.net.servicepointmanager.defaultconnectionlimit?view=netframework-4.7.2

There is a fix to change the throttling. Here is some code that can change it....

        /// <summary>
        /// Static values used for HTTP clients.
        /// </summary>
        public static void SetStaticServicePointManagerProperties(
            int p_DefaultConnectionLimit = 100,
            bool p_UseNagleAlgorithm = false,
            bool p_UnlimitedServicePoints = true,
            int p_MaxServicePoints = 1000)
        {
            // Default limit should be at least 10
            System.Net.ServicePointManager.DefaultConnectionLimit = p_DefaultConnectionLimit;


            // Whether to use nagle (delayed packet transmission)
            System.Net.ServicePointManager.UseNagleAlgorithm = p_UseNagleAlgorithm;


            // Optional limit on service points
            if (p_UnlimitedServicePoints)
            {
                // 0 means there is no limit to the number of System.Net.ServicePoint objects.
                System.Net.ServicePointManager.MaxServicePoints = 0;
            }
            else
            {
                // Specify the maximum
                System.Net.ServicePointManager.MaxServicePoints = p_MaxServicePoints;
            }
        }

Hope this helps anyone who may encounter the same problem.  At this point I am doubtful that the Progress open client API directly exposes the ServicePointManager's  "DefaultConnectionLimit".  (especially since that API was retrofitted quite a bit after PASOE came along).   

Keep in mind that the change affects all HTTP requests from the same .Net app, not just PASOE.

Posted by v205 on 05-Nov-2018 18:01

Wonderful. Thank you for your steadfastness in debugging this issue. This is valuable information.

This thread is closed