Which AppServer process is crashing things?

Posted by James Palmer on 01-Apr-2015 05:13

We've got a number of screens that send a query off to the AppServer. The AppServer builds the results into a temp table and then passes that back to the screen to display in a browse. From time to time our AppServers crash with a Java Heapspace out of Memory error. We have increased the heapspace to the maximum value we feel comfortable with due to resource issues on the server. But rather than just throwing more memory at the problem, is there a way we can easily trace which processes are generating so much data that it blows the limits? Is this something I can achieve with client logging on the AppServer? Or something of that ilk? I could visit all the pcode the AppServers run and add logging, but we're talking a few hundred programs so if I can do it in one swoop then it would certainly be easier! 

I realise that there are probably better solutions for our system that would negate the problems, but we're limited in terms of development time to throw at this sort of thing. 

Many thanks. 

Progress 11.2.1.

All Replies

Posted by Frank Meulblok on 01-Apr-2015 12:05

(Assuming stateless / state-free appserver here)

First, there's more going into the boker's Java heap than responses from agents. Also keep an eye out for number of client threads etc. (Especially if you're state-free and make heavy use of asynchronous procedures).

If you do have reason to believe it's the responses from the agents that blow the limit:

- Run requests that return Temp-tables/dataset parameters with the Temp-Tables log entry type active on the agent. (Good thing you're on 11.2, where that was introduced ;) ) That'll give you stats on number & average size of temp-table records so you can see if those are larger than expected.

- Run requests that receive or return MEMPTRs or LONGCHARs with the DynObjects.Other log entry type active on the agent. That should expose the size of those data types.

Posted by James Palmer on 02-Apr-2015 02:57

Sorry - yes it's Stateless.

Thanks for the info. I'll give it a try. I don't think we have any MEMPTRS or LONGCHARs passed around, but will double check.

Posted by James Palmer on 02-Apr-2015 05:55

This is probably a stupid question, but how do I activate client logging on the Agent? I've tried amending the startup parameters (and bouncing it of course) and it doesn't seem to be working.

Posted by Libor Laubacher on 02-Apr-2015 06:07
Posted by James Palmer on 02-Apr-2015 08:35

Thanks chaps. Got the logging enabled in test. What I'm struggling to understand now is that some temp tables I get info on what they contain, but on others I don't. Anyone care to comment? 

[15/04/02@14:29:25.164+0100] P-007068 T-006056 2 AS TEMP-TABLE     Created TEMP-TABLE stt-filters (ID:44 SHARED NO-UNDO Indexes:1) Mtsctlpm.p
[15/04/02@14:29:25.164+0100] P-007068 T-006056 2 AS TEMP-TABLE     Created TEMP-TABLE tt-MtsFilter (ID:42 NO-UNDO Indexes:1) Mtsctlpm.p
[15/04/02@14:29:25.165+0100] P-007068 T-006056 4 AS TEMP-TABLE     Statistics for TEMP-TABLE tt-ColourDefine (ID: 32):
[15/04/02@14:29:25.165+0100] P-007068 T-006056 4 AS TEMP-TABLE          Number of records: 56, Record size(B) Min : 28, Max: 49, Avg: 38
[15/04/02@14:29:25.165+0100] P-007068 T-006056 2 AS TEMP-TABLE     Deleted TEMP-TABLE tt-ColourDefine (ID:32) Mtsctlpm.p @ 481
[15/04/02@14:29:25.168+0100] P-007068 T-006056 2 AS TEMP-TABLE     Created TEMP-TABLE tt-ColourDefine (ID:72 NO-UNDO Indexes:1) stdplip.p @ 8882
[15/04/02@14:29:25.974+0100] P-007068 T-006056 2 AS TEMP-TABLE     Created TEMP-TABLE tt-MeterSiteBrowse (ID:43 NO-UNDO Indexes:2) CreateBrowseRecords Mtsctlpm.p @ 2174


Posted by James Palmer on 02-Apr-2015 08:37

As I post I realise that it seems to post the contents at the time of the delete. At what point is the table deleted?

Posted by Fernando Souza on 02-Apr-2015 08:45

Yes, the statistics get logged when the temp-table is deleted. Non-dynamic temp-tables are deleted when the program that defines them returns (or if it's a persistent procedure, when the procedure is deleted).

Posted by James Palmer on 02-Apr-2015 08:46

Thanks Fernando. What about dynamic ones?

Posted by Fernando Souza on 02-Apr-2015 09:13

Dynamic temp-tables have to be deleted explicitly via the DELETE OBJECT statement.

This thread is closed