PASOE - memory ownership

Posted by dbeavon on 10-Dec-2018 14:24

This probably should be a question with a simple answer, but the KB articles seem to be deliberately vague.  When an ABL session in PASOE is allocating memory (in classes, persistent procedures, dynamic objects, static datasets, etc) then that ABL session will be the exclusive owner of that memory, right?

Based on my experience, every ABL session has its own static members (on classes) and its own set of persistent procedures.  The OEE console is able to enumerate the memory associated with a given ABL session as follows:

Furthermore, one ABL session doesn't have responsibility, or capability, to maintain the memory of another session. Similarly an ABL session cannot maintain memory held outside of its session context.

 

I am trying to open a support case with Progress about memory in PASOE, but first I'd like to know who owns the memory.  We currently trim ABL sessions regularly (every hour) and it would seem to me that trimming the ABL sessions should implicitly trim any memory allocations that were initiated within those sessions.  Can someone please confirm?

 

Here are the KB articles I'm looking at.  They get progressively more helpful in their suggestions for finding leaks, but NONE OF THEM seem to say who owns the leaked memory.  If the memory is owned within the individual ABL sessions, then trimming the sessions should solve all our problems.

 

knowledgebase.progress.com/.../Detect-PASOE-memory-leaks-with-Dynamic-Objects-Logging
(11/6/2017)


knowledgebase.progress.com/.../New-ABL-Objects-Tracking-REST-API-under-PASOE
(5/29/2018)


knowledgebase.progress.com/.../Overview-of-the-new-11-7-3-PAS-for-OE-Watcher-monitoring-tool
6/27/2018

 

  

All Replies

Posted by Matt Baker on 10-Dec-2018 19:15

The second link about tracking ABL objects is probably where you should start.  Generally leaks like this are due to not deleting objects, or not deleting procedures and recreating them on each request.  As far as I know, the AVM doesn't track memory usage by individual object.

Posted by dbeavon on 10-Dec-2018 23:40

Thanks Matt.  My question is whether the deletion of the entire ABL session should serve in the place of deleting the individual objects and procedures within the session.    If the ABL session owns the memory and the session is trimmed, then the related memory (used for objects and procedures) should be released, right?

If the ABL session is trimmed, especially under normal circumstances, then it seems like the related memory should be cleaned out by Progress.  This seems reasonable, especially since I have never seen a KB article that describes what type of "clean-up work" is supposed to be done before trimming an existing ABL session.  Furthermore, PASOE has its own automatic mechanisms to trim inactive sessions after a period of inactivity.  This could be triggered at unpredictable times, without giving us a chance to execute our clean-up code.  It seems to me that Progress should then be responsible for doing all the clean-up for that ABL session.

Posted by Matt Baker on 11-Dec-2018 03:15

The memory is allocated to the session.  Some memory is shared with other sessions (parts of database connection information, shared procedure libraries and a few other things).   A session is really just a blob(s) of memory (this is a gross over simplification).  When a request comes in, a thread is assigned to the memory for the session, and the .p code is run within the context of that session. When the request completes, the memory (in state-free) is left more or less as-is, and the thread is freed back into the pool for further use by the next request that needs it.  If a session isn't bound to a thread, then nothing is happening with that session.  

Any unpredictability with shutdown is only such in relationship to the clock on the wall.  Shutdown is still done in an organized manner; shutdown procedures will run, database connections will be reaped and so on.  If your app is truly state free then it shouldn't matter to the app as a whole if some memory that hasn't been used in a while is freed back to the system.

A note here that memory free'd by an app, may still reside within the memory allocated to that app by the OS.  The OS may not immediately reclaim the memory.  The OS itself has heuristics based on memory pressure for when the memory is actually reclaimed for use by other processes.  Each OS is a bit different in how this is handled.  That is to say, terminating a session won't necessarily make the memory for the agent process shrink immediately (or at all).

knowledgebase.progress.com/.../pas-mproapsv-doesn-t-release-memory-after-clients-disconnect

Posted by dbeavon on 11-Dec-2018 04:26

Ok thanks.  Your first paragraph above answers most of my concerns.  I opened a support case for a problem where memory usage continues to grow at an uncontrolled pace, despite the fact that we only have about five abl sessions ever running at a time in an msagent process.  And even those few sessions are trimmed hourly and recreated on demand.

I suspect that our code may be using some type of abl handle or object or persistent procedure that isn't very well-behaved in the context of a passenger agent. (It may be a xml-dom-related handles, or file streams, or perhaps the jms adapter, if I had to guess.). I will assume that the expected behavior is that ALL of these types of things (and related memory) are supposed to be cleaned once the abl session is trimmed from the agent.  In other words the trimming of a session should serve the purpose of releasing memory, without requiring us to micro-manage every handle, object, or procedure.

To mitigate against ongoing memory leaks, I think it would be really nice if you guys did some additional/automatic housekeeping when the number of sessions in an agent went all the way down to ZERO. This happens multiple times a day in our environment, given that we are trimming inactive sessions so frequently.  It seems like you could essentially reset the entire agent process back to it's original state (as it was at the moment it started).  This would allow you to free up every possible resource that might have been orphaned along the way (and can't be attributed to any particular session).  

Posted by dbeavon on 11-Dec-2018 21:50

Today I learned that the CLR bridge (used in PASOE on windows) has quite a lot of memory that is shared in common with all the ABL sessions within an agent.  

I think this is an exception rather than a rule.  Putting the CLR bridge aside, I'm hoping that my ABL sessions will operate quite independently of each other..

More info:

community.progress.com/.../193386

This thread is closed