My Appserver Application is probably leaking memory very fas

Posted by waterfoul on 27-Jun-2012 15:30

After a few moments of heavy use using two servers the _proapsv.exe processes begin to consume an exobanant amount of memory (400MB each) and have a very large DBI file (800mb each) I'm going through the handles of each file that gets run with DynObjects.*:4 in the entry types, here are my findings:

File 1

5 handles

1. Dataset, does not get cleaned up, gets returned by the .p

2. Buffer, gets cleaned up properly

3. Temp-Table, does not get cleaned up, gets added to 1

4. Buffer, does not get cleaned up, is recieved from 1

5. Query, gets cleaned up properly

File 2

4 handles

1. Dataset, does not get cleaned up, gets returned by the .p

2. Buffer, gets cleaned up properly

3. Temp-Table, does not get cleaned up, gets added to 1

4. Buffer, does not get cleaned up, is recieved from 1

File 3

14 handles

1,2,3 Implicit Buffer, does not get cleaned up

4,5    Query, gets cleaned up properly

6,7    Dataset, does not get cleaned up, gets returned by the .p

8       Buffer, gets cleaned up properly

9       Implicit Buffer, gets cleaned up properly

10     Temp-Table, does not get cleaned up, gets added to 6

11     Implicit Buffer, does not get cleaned up, points to 11

12     Buffer, gets cleaned up properly

13     Temp-Table, does not get cleaned up, gets added to 6

14     Implicit Buffer, does not get cleaned up, points to 13

The only handles I don't see getting cleaned up are the 1, 2 and 3 in file 3 which are created on line 0 and are Implicit and those which are returned by (or are associated with those that are returned by) the .p which, to my understanding, shouldn't be cleaned up as they would be invalid handles when the .p is returned. Where am I loosing the memory?

All Replies

Posted by Admin on 27-Jun-2012 15:51

TABLE-HANDLE or DATASET-HANDLE parameters?

Dynamic widgets? Like Query Handles, Buffers Object Handles, etc.?

Posted by waterfoul on 27-Jun-2012 15:52

The returned datasets are dataset handles passed through a DATASET-HANDLE parameter. File 3 also recives 3 dataset handles. There are no other handles or widgets used besides those listed in the op.

Posted by Admin on 27-Jun-2012 15:56

Do you DELETE OBJECT the dataset handle parameters in the called procedure? The AVM will postpone the deletion untill after the procedure has finished (and the DATASET-HANDLE parameter was returned to the client).

Any sample code?

Posted by waterfoul on 27-Jun-2012 15:57

I don't DELETE OBJECT on any of the handles that are passed or the associated pieces, the problem is that the memory stays in use after the .p has returned and stays even when the app servers are idle.

Posted by Admin on 27-Jun-2012 16:01

That's pretty likely to be your problem...

When you are having a

DEFINE OUTPUT PARAMETER DATASET-HANDLE phDataset .

You need to DELETE OBJECT phDataset. Details are in the online doc - either on the DEFINE PARAMETER statement or the parameter passing syntax

Posted by waterfoul on 27-Jun-2012 16:02

Do I also need to do a DELETE OBJECT on the temp-tables and buffers added to the dataset?

Posted by waterfoul on 27-Jun-2012 16:07

I'm looking in the OpenEdge Help file (which I assume is just a local copy of the online help) and I can't see reference of what you are talking about

Posted by Admin on 27-Jun-2012 16:16

Just add the DELETE OBJECT statements for your output dataset-handle parameters if you cannot find it in the docs.

Or post sample code to illustrate your problem.

Posted by Admin on 27-Jun-2012 16:16

Do I also need to do a DELETE OBJECT on the temp-tables and buffers added to the dataset?

Not when you didn't create additional buffers or tables.

Posted by waterfoul on 27-Jun-2012 16:17

But if I created it in the application, even if its returned, it needs a DELETE OBJECT? If I can find what you are talking about in the manual I'll act ojn this but I am still not conviced that this is correct

Posted by waterfoul on 27-Jun-2012 16:18

I don't want to just assume you are right when nothing in the docs says otherwise since, at least the way I see it, it would make sense that when you delete an object it is no longer there and would be null or invalid when it is returned.

Posted by Admin on 27-Jun-2012 16:21

Always.

Posted by Admin on 27-Jun-2012 16:26

I don't want to just assume you are right when nothing in the docs says otherwise since

It's in the docs. You just didn't find it.

at least the way I see it, it would make sense that when you delete an object it is no longer there and would be null or invalid when it is returned.

As I wrote you earlier:

The AVM will postpone the deletion untill after the procedure has finished (and the DATASET-HANDLE parameter was returned to the client).

It doesn't need to assume logically. It needs to work, right?

Can't help you anymore if you don't show any code.

Posted by Admin on 27-Jun-2012 16:26

Just had a very quick look into the notes of the DELETE OBJECT statement in the online docs:

"When a temp-table object is passed as a TABLE-HANDLE output parameter, the deletion of the object following the DELETE OBJECT statement is postponed until the procedure goes out of scope. When the procedure returns, the TABLE-HANDLE is created, receives a copy of the original temp-table, and is then returned.

The OUTPUT TABLE-HANDLE parameter creates a TEMP-TABLE in the target procedure, which is added to the SESSION widget-pool. You must use the DELETE OBJECT statement to delete this TEMP-TABLE when it is no longer needed, or you will create a memory leak."

The same applies to DATASET-HANDLE parameters.

OK?

Posted by waterfoul on 27-Jun-2012 16:33

OK but how do I know this is actually how things work (as is intended by the progress developers) and not just how it appears to work? I don't like programming based on "rules" which aren't really there and are subject to change. Also is this how the temp-table handles behave which are added to the dataset or could the garbage collector theorettically clean those up before the dataset is returned and cause the dataset to have missing data? Also when it says "is no longer needed" does that mean "is no longer needed by this procedure" because the handle is needed after the procedure exits

Posted by Admin on 27-Jun-2012 16:41

OK but how do I know this is actually how things work (as is intended by the progress developers) and not just how it appears to work?

Training, Documentation, Developer conferences (like pug-challenge.us or pug-challenge.eu), this forum.

I don't like programming based on "rules" which aren't really there and are subject to change.

Those rules have been there from day one of the TABLE-HANDLE/DATASET-HANDLE parameter. Probably more than 10 years now. I don't think they will ever change.

Posted by waterfoul on 27-Jun-2012 16:46

If it is not documented then there is no reason why it isn't subject to change but I guess I will assume (Like everyone else) that it won't.......

Posted by Admin on 27-Jun-2012 16:51

Believe me. This is documented.

But it's too late for me now. You gotta find someone else to point you to it.

Posted by waterfoul on 27-Jun-2012 16:51

I just tried it with cleaning up all tables after they were added to the dataset and now I get no data back....

This is how I added the table

/* create a temp table like the actual table and set it up */

CREATE TEMP-TABLE bfr.

bfr:CREATE-LIKE(Cur).

bfr:TEMP-TABLE-PREPARE(tbl).

outputData:ADD-BUFFER(bfr).

DELETE OBJECT bfr.

and then I did

outputData:GET-BUFFER-HANDLE(tbl).

That did not work.... It only would work if I deleted the DELETE OBJECT bfr Line so apparently you are supposed to delete all of the objects except the actual temp table objects

Posted by Admin on 27-Jun-2012 17:01

I just tried it with cleaning up all tables after they were added to the dataset and now I get no data back....

I offered you a couple of posts back to review some code.

Posted by rbf on 28-Jun-2012 02:37

FYI there is no garbage collection for 4GL objects like buffers, temp-tables and datasets. GC applies to class objects only.

These are the rules:

1. when you CREATE an object, DELETE it

2. when you receive a temp-table or dataset using OUTPUT TABLE-HANDLE/DATASET HANDLE, you must DELETE it in the target procedure as well

These rules *are* in the docs, but they are hidden well.

Tip: use the FINALLY statement for this whereever you can as in that case the cleanup code even fires after error conditions.

HTH,

-peter

Posted by Peter Judge on 28-Jun-2012 10:31

waterfoul wrote:

I just tried it with cleaning up all tables after they were added to the dataset and now I get no data back....

This is how I added the table

/* create a temp table like the actual table and set it up */

CREATE TEMP-TABLE bfr.

bfr:CREATE-LIKE(Cur).

bfr:TEMP-TABLE-PREPARE(tbl).

outputData:ADD-BUFFER(bfr).

DELETE OBJECT bfr.

and then I did

outputData:GET-BUFFER-HANDLE(tbl).

That did not work.... It only would work if I deleted the DELETE OBJECT bfr Line so apparently you are supposed to delete all of the objects except the actual temp table objects

You don't need to delete the temp-tables if they are part of a ProDataSet (usually). You must delete the ProDataSet before returning. It should (probably) be the last thing you do in the procedure, as Peter (the other one) recommends.


DEF OUTPUT PARAM DATASET-HANDLE phDataset .

CREATE DATASET phDataset.

/* complete building of dataset, get data, do stuff */

RETURN.

FINALLY:

  /* will ensure that you have no leak on the AppServer. Don't worry, the data will get back to the caller */

  DELETE OBJECT phDataset.

END FINALLY.

Take a look at the logging features (LOG-MANAGER) in the ABL and AppServer (in the doc too). You can see that the phDataset above gets a state of DELETE PENDING, which means that it's basically in transit to the client/caller.

If you want official confirmation of this behaviour, contact Tech Support. That way, if it doesn't exist in the formal documentation set, it can be logged as a doc bug and added in a future release. But as everyone here says, that's the behaviour as it stands today, and the language tends to be extremely backwards compatible, so you can code as suggested with confidence.

-- peter

This thread is closed