Memory leak in OpenEdge.Net.HTTP classes?

I don't see any DELETE OBJECT references in your code. You should really be cleaning up after yourself, especially when using .NET.

Eventually when your OE objects get cleaned up the .NET GC will kick in. But try not to depend on GC if you want the most optimal memory usage.

But I am not using .NET, these are Openedge classes available in Openedge.net.pl.

Well that is unfortunate naming then :)

But good to know Peter. Thanks.

Anyway, just tried adding a delete object for oRequest and oResponse inside the FOR EACH block with same result, 248M at the end of the execution.

Thanks Peter,

I have executed the code with logging for dynobjects. But once I got that log file, do you guys have any tool to validate what has been created/allocated against what has been destroyed/unallocated?

Just my fault for assuming that NET meant one of the classes that actually called .NET.

I suggest we just blame Microsoft :)

See this KB for some sample code to parse the log

knowledgebase.progress.com/.../P133306

I would say fluent syntax might be the problem here, there is probably a client instance created and you have no reference to it

ClientBuilder:Build()

Well, I had to change quite a few things to that program to get it to work with the format I was getting the log into it, plus I have addded  processing to compare allocate vs deallocate memory in longchars/memptrs (not sure how correct is my processing there). But finally there is a result:

My findings are:

 There are 58290k that remains allocated that in the log file are logged as handle 0 without an address (not sure what that means or if my processing is correct).

  Procedure sockethelper remains in memory.

  ParameterList object from DefaultRequestBuilder remains in memory.

 JsonObject from JsonBodyResponseFIlter remains in memory.

 There are several instances of allocated memory in ByteBucket (quite a lot) that never gets deallocated.

 There is one instance of allocated memory in ClientSocket that never gets deallocated.

It looks to me that there is somethin wrong there, but let me know, also if possible could you email me how to submit a bug?

Cheers,

PS. I have attached here my modified version of leakcheck.p in case is of anyone's interest.

Not sure to understand what the problem is, whatever using fluent syntax or named instances shouldn't change the behaviour of the GC, should it?

And it should be the reponsability of the classes to delete whatever it is that needs to be deleted once the GC collects my instances, not mine as I don't need to know the internals of the library to use it.

Agreed, I just don't trust GC that much although it's getting better from what I've heard :)

I expect if the GC kicks in and delete the objects fluid should work just fine and any allocated memory to be freed by the object that allocated it.

GC is always a little flaky in every language I have used. Even though this should work as is... it would be interesting to see what happens when you move all of the HTTP logic into a different procedure (new .p file) and run that code from inside the customer loop.

Either it behaves the same and shows a true memory leak or it doesn't and OE has a GC/bug issue.

Tried that, moving the request calls to an independent procedure, same result, tried with a get operation instead of a put (just in case) same result but less memory used, I am pretty sure there is a memory leak there.

Could anyone point me to where can I submit the bug?

Also, is there someone else using these classes in a different way where that works around this memory leak? It seems surprising nobody has noticed, it leaks almost 0.25Mb for every call for me...

There's at least one leak caused by circular references held by the ClientSocket class, incidentally.

I've logged issue PSC00333868 for this problem. Contact TS if you want your name associated with it (for notifications etc)..

[apologies for the long post. the TL;DR is there were a couple of leaks, which are fixed in 11.6 and also be careful of reading too much into the log-manager's outout]

After some investigation I found 2 styles of leak

- the circular one I mentioned earlier, where  a .P is run persistently from an object and which holds a reference to the object, thus rendering garbage collection impotent.\

- i like to use a Initialize/Destroy pattern instead of trusting a constructor to do everything; this requires someone to call the Initialize method on object construction (done) and the Destroy method when the object is being destroyed. This wasn't always done.

The log-manager's output from DynObjects.* - especialy with a high enough log level - needs to be carefully read

- Entries that refer to LONGCHAR only, like the line below are intended for internal-to-ABL-core developers for their edification and delight (and not for ours). They can be safely ignored.

PutBytes OpenEdge.Core.ByteBucket @ 193    x166750    LONGCHAR    19

- Sometimes, the AVM creates objects internally from non-object arguments you pass it. The line below

NewRequest OpenEdge.Net.HTTP.DefaultRequestBuilder @ 54    0    Progress.Lang.Object    Progress.Lang.ParameterList

is the result of the following ABL

oRequest = dynamic-new string(oRequestType:TypeName)( GetOptionStringValue('method':u),
                                                          cast(GetOptionObjectValue('uri':u), URI)).
                

The AVM changes the 2 arguments into a Progress.Lang.ParameterList object over which you have no control. You have to take it on faith that that object will be "GC'ed"

- Objects that are 'held' by static members will only show up as created and not deleted. For instance, the following code

    /** Registry for mapping build types to their implementations */
    define static public property Registry as BuilderRegistry no-undo
        get():
            define variable oRegistry as BuilderRegistry no-undo.
            if not valid-object(RequestBuilder:Registry) then
            do:
                assign oRegistry = new BuilderRegistry().
                RequestBuilder:InitializeRegistry(oRegistry).                
                assign RequestBuilder:Registry = oRegistry.
            end.
            return RequestBuilder:Registry.
        end get.
        private set.

shows up in the logs as

propGet_Registry OpenEdge.Net.HTTP.RequestBuilder @ 74    1049    Progress.Lang.Object    OpenEdge.Net.HTTP.BuilderRegistry

However, there are no corresponding Deleted-by-GC entries. This is because of the fact that the references are held by static members which are destroyed by the AVM when the session shuts down; this destruction happens in an purely internal way, and so there are no logs of the event.

hth

Hi Peter,

Thanks for your answer, it will prove very helpful to understand the log-manager's output.

One thing I noticed was that in the destroy method, the persistent procedure is deleted with a no-error clause, and every time that destructor was called (I debug that bit) the handle of the procedure was not valid, not sure if that adds any extra useful information, if not, just ignore it.

Cheers,

Peter was there ever any resolution to the memory leak in OpenEdge/Net/HTTP?

We just deployed PASOE to production for the first time and our MS agents are growing to use a lot of resident memory (well over 1 GB after just a couple hours).  These MS agent processes only have about 5 ABL sessions in each of them.  When I click the sessions and examine the "stack" data that is associated with each, it lists objects and persistent procedures that are occupying memory.  I see a lot of (non-STATIC) references in each session to "BuilderRegistry.r".  

There is usually 1 each of a variety of "STATIC" classes, and those don't bother me.  But the "BuilderRegistry" entries have arbitrary numbers for the "ObjectId" and there are lots of them! Image above.

I will open another ticket with tech support but I was hoping maybe I could shortcut that process if somebody else had encountered this already.  We are running PASOE on windows with OE version 11.7.4.

Thanks, David

OK, thanks for the explanation.  I was originally thinking that any objects showing up that didn't have the "STATIC" notation was an indication of a leak - especially if there were a large number of them.  But I didn't do much digging yet.  Now that you say this, I've noticed that my own singleton classes (listed with a "STATIC " notation) are always paired with one that does *not* have that same notation.  I suspect the "STATIC" one represents the type, and the one without that notation is the singleton instance of the type.

Yes, my screenshot was from the "stacks" link in the OEE console for PASOE. (The stacks link can be found on the individual sessions within the "agents" of an abl application).