Best practise regarding App Server connections

Posted by ojfoggin on 26-Jun-2009 08:08

I've been given the task of trying to optimise our system and try to iron out any parts that are causing excess network traffic or CPU timings etc...

One thing we are now wondering about is best practise regarding App Server connections.

At the moment the system works by creating a connection to the AppServer each time it requires data and then runs a procedure in the AppLibrary in order to get the data and then closes the connection.

I don't know why it is this way (I've only been around for around 7 months) but I'm wondering about doing it differently.

Is it better to open the connections to the AppServer libraries when the client is launched and then keep them open all the time (passing the handles around to programs that need them, or even using shared handles) and then when the client is closed closing all of the AppServer connections.

This way we only open them and close them once and can access them from any part of the software.

The only thing is that we will potentially have 1000 AppServers connected at the same time for long periods of time.  Does progress manage this efficiently or will this kill the server?


I guess my question is this...

Which is better, having connections open for very short amounts of time and having to connect/disconnect hundreds of times every hour or having a fairly large number (circa 1000) connections open but remaining open and accessible at any time.

Thanks for any advice.

Oliver

All Replies

Posted by rbf on 26-Jun-2009 08:23

Hi Oliver,

Assuming that you are using a Stateless AppServer, it is certainly much more efficient to connect once and issue calls during the life time of your client session. It effectively reduces your number of calls by a factor 3. Do not worry about 1,000 simultaneous connections. These are not taking up any resources between calls. They are administered by the broker. Read up on the AppServer documentation to find out more.

If you are using a State-reset AppServer it is a different story, because then your application is probably depending on the automatic cleanup performed between calls.

If you are using a Statefree AppServer, there is no such thing as connect or disconnect, each call inherently connects and disconnects.

Posted by ojfoggin on 26-Jun-2009 08:25

Thank you,

We are using a stateless server.

In fact, I have just been provided with a (very old) analysis document of our code and it mentions the app server usage.

I may have to get this pushed through development then.

Thanks for your help!

Oliver

Posted by ojfoggin on 26-Jun-2009 10:05

Well, that was a hell of a lot easier than I thought it would be.

We have a load of procedures that connect and disconnect their own app servers and now we have changed the publish code so that they all use the same session app server.

Less than an hour!

Posted by rbf on 27-Jun-2009 06:11

Great! Let us know what improvements you are measuring now.

-peter

Posted by ojfoggin on 01-Jul-2009 07:02

At the moment we are having a few teething problems but I think they are coming over from bad practise by previoud developers.

I have now set up the system to have one app server wet up on login and then disconnect it at logout and anything in between uses the same appserver (unless the appserver gets stopped and started and then it reconnects again).

The problem that we are having is that the appserver is restarting itself with the error message...

"(Procedure: 'libapp/ordlib1.p' Line:0) SYSTEM ERROR: Attempt to define too many indices. (40)"

I have a feeling that this is caused by super procedures not being closed down properly when exiting library files.

We have a huge library file called ordlib.p (Order library) and this then has 2 super procedures called ordlib1.p and ordlib2.p.  I think I have fixed the closure of these super procedures but I don't know any way to test it.

Is there a way to see the current number of temp-table indices on the appserver?  That way I can see what the numbers are doing when I apply the changes.

Thanks

Oliver

Posted by GregHiggins on 01-Jul-2009 10:00

If you have session super procedures on an AppServer, they should generally be run once during startup.

No other procedure should start these supers without knowing they are needed.

session:super-procedures will return a list of string(handles) of current session super-procedures.

session:first-procedure will return a starting handle for chain of current persistent procedures.

If you have a procedure handle, next-sibling is the next persistent procedure in the chain.

When I create a session super I do so using

{myself.i &name="path/to/me/pgmname.p" }

subscribe to "{&name}" anywhere run-procedure "myself":u.

procedure myself:

  define output parameter opMe as handle no-undo.

  assign

    opMe = this-procedure

    .   

end procedure /* myself */.

/* end of myself.i */

Then if I want to know if I need to run path/to/x.p, I simply

publish "path/to/x.p" ( output handle-var ).

if not valid-handle ( handle-var ) then run "path/to/x.p" persistent. ...

Posted by rbf on 04-Jul-2009 03:55

ojfoggin wrote:

Is there a way to see the current number of temp-table indices on the appserver?  That way I can see what the numbers are doing when I apply the changes.

The attached program allows you to monitor buffers, persistent procedures and memory leaks in general on the AppServer.

Run getappsmeminfo1.p on the client. If there is an AppServer connection available, it will fire up getappsmeminfo2.p on that AppServer.

Set your max number of agents temporarily to 1 or each call will end up in a different agent which would make it nearly impossible to interpret the results.

Posted by Admin on 04-Jul-2009 15:33

Is there a way to see the current number of temp-table indices on the appserver?  That way I can see what the numbers are doing when I apply the changes.

Another good (and often overseen) feature when working with the AppServer is the Progress Debugger. The debugger is able to attach to an AppServer session and monitor dynamic objects that are used. You can turn on the "Monitoring of dynamic objects" and when you turn of that option you'll see exactly what objects have been instantiated by which procedure.

Won't hlep directly with the number of indexes as you'd need to count them manually, but you'll see all the temp-tables.

Works also best when you limit the number of AppServer processes to 1.

Posted by rbf on 05-Jul-2009 08:49

> Another good (and often overseen) feature when working with the AppServer is the Progress Debugger. The debugger is able to attach to an AppServer session...

That is true and I have used it many times in the past. However, in recent versions 10.1C and 10.2A I have often not been able to use the attachable debugger anymore. I have logged several calls for this but they all die on the issue of reproducing. Are you still using the attachable debugger in 10.1C and 10.2A without any issues?

Posted by Admin on 05-Jul-2009 15:20

Are you still using the attachable debugger in 10.1C and 10.2A without any issues?

Yes (and no )

Works fine with 10.2A01 on Vista 32 bit. Haven't tried 10.1C recently, but I don't remember any issues there.

Doesn't work on Windows 7 64 bit. Don't know exacly why. I can attach from within an XP VM to the host, but can't attach from the Windows 7 host directly (to the same appserver).

Posted by Admin on 05-Jul-2009 15:53

I had need of the debugger recently and couldn't get it going under 10.1C01.  Tech support couldn't help either.

Posted by ojfoggin on 06-Jul-2009 02:56

Thanks for all the help guys!

I will have to wait until I get the go ahead from my boss (he's on holiday for a week) but I am going to spend a week redesigning how the AppServer works.

Although it will take a lot of work it shouldn't be too bad as most of the startup/shutdown stuff is done in include files.

Anyway, aside from that I have one final question.

Using the startup paramter on the AppServer, how do I start a session super procedure?  We have procedures in the connect and disconnect but I'm not sure how to specify that they should be SSPs int he startup parameter.

Thanks again

Oliver

Posted by rbf on 06-Jul-2009 03:28

ojfoggin wrote:

Anyway, aside from that I have one final question.

Using the startup paramter on the AppServer, how do I start a session super procedure?  We have procedures in the connect and disconnect but I'm not sure how to specify that they should be SSPs int he startup parameter.



In the Advanced Features -> Startup box you specify a procedure that contains code like the following:

RUN mySuper.p PERSISTENT.

mySuper.p in turn contains the following line:

SESSION:ADD-SUPER-PROCEDURE(THIS-PROCEDURE).

Posted by ojfoggin on 06-Jul-2009 03:29

Thanks again.

I'll let you know how it goes.

Posted by GregHiggins on 06-Jul-2009 04:29

What you need to look out for is client (not, generally, appserver) code which looks like this:

RUN something.p PERSISTENT set aHandle ON  ...

That is code which binds the AppServer to the client.

The RUN ... PERSISTENT ... ON ... syntax cancels the statelessness of an AppServer agent (session) and binds it to the client until the procedure is deleted or the client disconnected.

When the code connects and disconnects frequently, as has been done in the past, the agents unbind when the client disconnects and return to their stateless state without any additional client programming.

I've been writing stateless AppServer code the early days of V9 and I have never used the RUN ... PERSISTENT ... ON ... syntax, so if you don't find the construct in your code, I would consider that a good thing. If you do find it, you will need to ensure that those proxy procedures get deleted and don't keep the AppServer agent bound to the client unnecessarily. Part of me wants to assume that's what you meant by "I have sorted out closing all the persistent procedures",  but it needs to be said.

Posted by ojfoggin on 06-Jul-2009 04:33

Thankfully that was the easy part.

The client has a Super Proc that subscribes to "PublishAppLoadRun" which used to do what you are saying.

By changing the Super Proc code I was able to change the entire client software in one go

The problem I have now is that the infrastructure of the rest of the software has not been written in order to deal with HouseKeeping tasks etc...

That's what I'm having to do now

Posted by rbf on 09-Jul-2009 16:10

Concerning the attachable debugger, I figured out my problem with Tech Support and now it works. It seems that nowadays three steps are needed when you want to attach the debugger to a remote machine:

1. ProDebugEnable the remote machine

2. Debug Enable the remote process

3. ProDebugConfig the remote process

This last step was new to me but it works now.

Posted by GregHiggins on 09-Jul-2009 16:35

-debugReady no longer works ?

Posted by rbf on 11-Jul-2009 08:18

greghiggins wrote:

-debugReady no longer works ?

Yes I should have mentioned that you can replace step 3 by setting the -debugReady parameter in your AppServer agent.

In that case you either have to use -debugReady 0 and look up the assigned debug port in your AppServer log file or use -debugReady and be very certain that you every only start a single agent. With -debugReady 0 you should of course also ensure that you only start a single agent but the good news is that the assigned port numer will generally be 9999.

Too bad the attachable debugger does not remember the last used host name and port number.

Posted by ojfoggin on 15-Jul-2009 03:37

Calling super code.

I'm just trying to rewire some of our library apps.

lib.p has 2 SPs lib1.p and lib2.p

At the moment (for example) ...

lib.p will contain a procedure called foo but all it contains is "run super".

Then lib1.p will contain the actual code for foo.

The client will have the code...

[...run lib.p persistent set lh-lib...]

run foo in lh-lib.

[...end lh-lib...]

Now, do I need foo to be in lib.p in order for it to run in lib1.p?  I'm thinking not but I just wanted to make sure.  Surely by having lib1.p as a SP of lib then if lib can't find foo it will automatically go through lib1.p and then lib2.p until it finds it?

Thanks

Oliver

Posted by Stefan Drissen on 15-Jul-2009 15:45

Yes, tested with the following mini-code:

c.p

PROPATH = "c:\local".

DEF VAR hp AS HANDLE NO-UNDO.

RUN lib.p PERSISTENT SET hp.

RUN foo IN hp.

lib.p

DEF VAR hp AS HANDLE NO-UNDO EXTENT 2.

DEF VAR i AS INT NO-UNDO.

DO i = 1 TO 2:

   RUN VALUE("lib" + STRING(i) + ".p") PERSISTENT SET hp[i].

   THIS-PROCEDURE:ADD-SUPER-PROCEDURE(hp[i]).

END.

lib1.p
PROCEDURE foo:
   MESSAGE PROGRAM-NAME(1) VIEW-AS ALERT-BOX.
END PROCEDURE.
lib2.p
PROCEDURE foo:
   MESSAGE PROGRAM-NAME(1) VIEW-AS ALERT-BOX.
END PROCEDURE.
Do note that if foo exists in both lib1.p and lib2.p (as above) the order in which they are added as super-procedure counts, the last program added to the super stack containing foo will be run.
BTW I am really enjoying your line of questioning. We have had various similar things to ponder in our recent transition from c/s to AppServer.

This thread is closed