Pacific Appserver Performance 11.6

Posted by carl.williams on 05-Feb-2016 02:59

Has any one done any independent tests comparing the classic Appserver the OEPAS for 11.6. I have noticed a few things from a simple test on a windows PC running PAS and on Linux:

- No application metrics available (logged with support)

- For PAS development mode 5 sessions are always started even if setting "max abl sessions per agent" to 1. What then happens is that only 1 of these sessions will be used

- Intermittent error with the progress.cfg file being locked thus stopping creation of new pas instance in OE Explorer with Java sevlet error. This occurs if creating/deleting PAS instances through OE explorer multiple times (logged with support). Stopping & starting adminserver sorts this.

- The same agent session is always used if available is this expected behaviour. 

The classic appserver jumped about between agents but PAS will always use the same session if available from any number of different sessions. So in effect the one session could be used 90% of the time and the others very rarely.If this sessions fails due to a DB connection lost or for any other reason this would I suspect stop the appserver from working. If there are any memory leaks or the like then the one session will suffer from a build-up of problems rather than being shared out.

- Performance appears to be less than the classic appserver: 63 vs 24 seconds on a test I ran with an empty test.p file

ETIME(YES).
DO i = 1 TO 100000:
    RUN test.p ON hServer.
END.
MESSAGE ETIME / 1000 VIEW-AS ALERT-BOX INFO BUTTONS OK.

- Repeating the test with 100,000 calls running a test.p with 1 record temp-table then Classic wins 32 vs 120

- Repeating the test with 1000 calls running a test.p with 100 record temp-table very similar.

- Repeating the test with 1000 calls running a test.p with 1,000 record temp-table then PAS wins 14 vs 19

- Repeating the test with 1000 calls running a test.p with 10,000 record temp-table then Classic wins 39 vs 37

They do use a different executable is it possible they cache temp-tables, etc differently or that the PAS appserver session runs the .p faster?

- Finally a simple question about how do we trim sessions for the agent.

All Replies

Posted by Irfan on 05-Feb-2016 07:52

Hi Carl,
 
Please find the answers inline
 

Posted by Irfan on 05-Feb-2016 07:57

Has any one done any independent tests comparing the classic Appserver the OEPAS for 11.6. I have noticed a few things from a simple test on a windows PC running PAS and on Linux:

- No application metrics available (logged with support)

Were you looking for a metrics like localhost:16680/.../metrics

- For PAS development mode 5 sessions are always started even if setting "max abl sessions per agent" to 1. What then happens is that only 1 of these sessions will be used

If you want to start only 1 session, then you have to set two properties

   maxABLSessionsPerAgent=1

   numInitialSessions=1

- Intermittent error with the progress.cfg file being locked thus stopping creation of new pas instance in OE Explorer with Java sevlet error. This occurs if creating/deleting PAS instances through OE explorer multiple times (logged with support). Stopping & starting adminserver sorts this.

Should be a bug

- The same agent session is always used if available is this expected behaviour.

The classic appserver jumped about between agents but PAS will always use the same session if available from any number of different sessions. So in effect the one session could be used 90% of the time and the others very rarely.If this sessions fails due to a DB connection lost or for any other reason this would I suspect stop the appserver from working. If there are any memory leaks or the like then the one session will suffer from a build-up of problems rather than being shared out.

In PASOE, if a session is started and used while executing any client request then the same session will continue to be used if only 1 client request is sent overall. The behaviour is different to Classic Appserver as in Classic it tends to use all the started agents. In PASOE, a new session is used if it is only required to.

- Performance appears to be less than the classic appserver: 63 vs 24 seconds on a test I ran with an empty test.p file

ETIME(YES).

DO i = 1 TO 100000:

   RUN test.p ON hServer.

END.

MESSAGE ETIME / 1000 VIEW-AS ALERT-BOX INFO BUTTONS OK.

Performance gains on using PASOE can be observed based on load(concurrent clients & payloads). This way, you should be able to see the benefits of running a Multi-Session Agent.

- Finally a simple question about how do we trim sessions for the agent.

. The classic appserver jumped about between agents but PAS will always use the same session if available from any number of different sessions. So in effect the one session could be used 90% of the time and the others very rarely.

As mentioned to you earlier, PASOE uses sessions based on number of client requests. If only 1 client request is executed, then there is no need for running a different session everytime.

Considering this fact, there might be below reasons why a session has to be killed

1. Know if there is hung Client requests and terminate them  - /oemanager/applications/<AppName>/clients?timeoutms=100000

2. Terminating any client session - /oemanager/applications/<AppName>/sessions?sessionID=&terminateOpt=1

3. Deleting free ABL Sessions in the Agent – in JMX SessionManager -> terminateFreeABLSessions

4. Terminate a given ABL session – We are working on this one, should be available soon

Posted by carl.williams on 12-Feb-2016 07:30

Thanks for the response.

Posted by Blake Stanford on 18-Feb-2016 09:22

Irfan - Would you elaborate on how one is going to see performance gains based on load?

I have been running some performance tests trying to determine if we want to migrate to PAS for OE.  I have found that PAS OE does not perform as well as the Classic AppServer and uses more memory.  

From our Windows GUI client we connect directly to a State-reset and/or State-free AppServer

"-URL AppServerDC://<host>:<port>/DevelAppSrvrStateReset -sessionModel Session-managed"

"-URL AppServerDC://<host>:<port>/DevelAppSrvrStateFree -sessionModel Session-free")  

Below are some of the tests I ran.  

Windows 11.5/11.6 GUI client with PASOE v.s. Classic AppServer on a UNIX machine.

To emulate the State-reset AppServer in PAS OE, one must add a couple of lines to the connect event procedure to bind the session and then unbind it and reset it in the disconnect procedure.    

Average time over 1000 iterations of each test (milliseconds):

Test1 - 1 Client Connect to an AppServer, run a procedure on the AppServer passing a 100 record temp-table as an input parameter, do nothing else and return.

Classic State-Reset - 14.889

Classic State-Free - 13.058

PAS Session-managed bound state-reset - 18.65

PAS Session-free unbound - 17.89

Test2 - 1 Client Connect to an AppServer, run a procedure on the AppServer passing a 100 record temp-table as an input-output parameter, do nothing else and return.

Classic State-Reset - 17.248

Classic State-Free - 16.101

PAS Session-managed bound state-reset - 21.68

PAS Session-free unbound - 21.04

Test3 - 1 Client Connect to an appserver, run a procedure open a query against database, build a temp-table from the query (141 records) and pass the temp-table back:

Classic State-Reset - 136.9

Classic State-Free   -  136.6

PAS Session Managed bound state-reset - 172.9

PAS Session Free unbound - 140.5

Memory allocated after tests:

                                                                                  Instance/Broker   Servers/Multi-Agent Servers       Total

                                                                                        (java.exe)       (_proapsv.exe/_mproapsv.exe)

Classic State-Reset -                                                   251356k                        192536k                            443892k

Classic State-Free -                                                     557904k                        188524k                             746428k

PAS Session Managed bound state-reset -            656092k                        193916k                            850008k    

PAS Session Free unbound -                                    656896k                        154860k                             811756k

Posted by Irfan on 18-Feb-2016 10:20

Hi Blake,

Considering the fact of a Production environment which might have many concurrent clients connecting to the appserver with various loads, the kind of tests that we ran has of load of 25 concurrent clients running on 25 different sessions which has 25 connections under 1 Multi-session Agent.

The payloads that we used for testing looks like this for each transport

APSV – 400 KB for each Client( 100 temp-tables where each temp-table has around 200 fields)

REST – 260 KB of json payload for each Client

WEB - 2.7KB to 25KB of payload for each Client( ran for 25 and 50 concurrent clients)

The multi-session agents takes up more memory at start-up, so if you compare the memory of 1 _proapsv process with another _mproapsv process then it might not be the right comparison. The comparison should be between "n" _proapsv processes vs 1 MS-Agent with "n" sessions, where "n" refers to the number of "Classic Agents/MS-Agent sessions" you want to use in the appserver. Again the same applies to the Classic Appserver broker and the PAS Server.One PAS Server has the ability to run various web-applications inside it, support all kinds of transports for both Session-free and Session-Managed applications, manager applications and others whereas if you need to do the same in Classic Appserver you have to do it by configuring many servers. So the initial memory loaded in the PAS looks bigger, but the resource utilization of it should be considered once you try to put a decent load on the server

Posted by dbeavon on 18-Feb-2016 11:16

Is there a link that shows your results with the comparisons (PAS vs legacy appserver)?  And can you give us the testing repros that you used - so that we can run the same performance tests you did?

If we are going to hope for an improvement from PAS, we might as well start by trying the same repros (the ones where you've already demonstrated an obvious performance boost).  Then we can move on from there and see how our own workloads compare.

A lot of people would probably be interested in trying out your new PAS, provided it was an improvement for the types of workloads we have.  We do high volumes of appserver-based reports, and I suppose that would be analogous to the type of PAS testing that you described.  I'd be interested in doing a PAS P.O.C.... But it would be nice if we could start with some encouraging comparisons right off the bat (like the comparisons you had described).

Posted by Blake Stanford on 18-Feb-2016 11:41

Irfan,

Okay, l misstated the memory consumption results. The memory averages were from a Multi-session Agent with 5 sessions and the combined memory of  5 Classic Agents after running the various tests (not at startup).

Are you saying that a Multi-session agent with N sessions will consume less memory than the same number of  Classic Agents or that in total PASOE should consume less memory because one might need fewer MS-Agent sessions than Classic Agents?  If the latter, then wouldn't we would need to move away from using bound connections thus introducing the additional overhead of context management?  

I also ran tests with 5 concurrent clients running on 5 different sessions and found the performance results to be the virtually the same Classic always outperformed PASOE.  I'm not sure I understand how putting more load on will change the performance?

Note: These are somewhat primitive tests with out-of-the-box tuning.

Thx,

Blake

Posted by Irfan on 18-Feb-2016 12:07

Please find my comments below

Okay, l misstated the memory consumption results. The memory averages were from a Multi-session Agent with 5 sessions and the combined memory of  5 Classic Agents after running the various tests (not at startup).

Okay. Can you put down the amount of memory used by the resources for running Classic Appserver(Adminserver, nameserver, Appserver broker, number of agents)  and then PASOE(Server and MS-Agent)

Are you saying that a Multi-session agent with N sessions will consume less memory than the same number of  Classic Agents or that in total PASOE should consume less memory because one might need fewer MS-Agent sessions than Classic Agents?  If the latter, then wouldn't we would need to move away from using bound connections thus introducing the additional overhead of context management?  

I am referring to your first statement, which is Multi-session agent with N sessions will consume less memory than the same number of  Classic Agents. In Classic each agent process gets started which holds some memory where as in MS-Agent only 1 Agent is used with a set of sessions.

I also ran tests with 5 concurrent clients running on 5 different sessions and found the performance results to be the virtually the same Classic always outperformed PASOE.  I'm not sure I understand how putting more load on will change the performance?

I haven't ran with 5 concurrent clients earlier, I always used 25 or more. I will give it and try at my end and see what I get.
May I know what your appserver code does

Also, what are your Agent configurations for number of sessions and Agents

Note: These are somewhat primitive tests with out-of-the-box tuning.

Posted by carl.williams on 22-Feb-2016 03:22

So are we saying that the Progress performance tests are comparing one 25 session MUAgent vs 25 x Webspeed, 25 x Webservices, 25 x REST & 25 x Appserver agents to support the same clients assuming maximum of 25 mixed requests concurrently? In other words only if we need to support all 4 transports concurrently would we see any benefit?

So for instance if it were purely a classic appserver handing ABL code vs PAS handling only ABL code that there would not be the same benefits?

Posted by Bill Wood on 22-Feb-2016 06:16

Progress does lots of tests in lots of environments....  You definitely don't need to support all 4 transports concurrently to see a benefit.

Your last point though hits the nail on the head, partially:   Ultimately both PAS for OE and Classic AppServer are AVMs (ABL Virtual Machines) that execute ABL byte-code.  The byte-code is the same, and if the machine is identical, and is CPU bound, then the speed of executing a line of ABL won't matter.  

But the AVM is limited by the resources on a machine.  As you use more memory, more disk IO and more CPU you will start running into limitations on a machine.  This is why you will see improvement under load with PAS for OE because it uses less resources overall for the same amount of 'raw' ABL processing power.  In other words, a similar sized machine will be able to handle more requests faster with PAS for OE than Classic AppServer.   Or, a machine handling the same number of requests will use fewer resources with PAS for OE than Classic.  The ABL byte-code is processed the same way, but there is less contention for scarce resources, and less swapping/switching with the new architecture.

Posted by Tim Kuehn on 22-Feb-2016 06:34

I was under the impression that PASOE enabled one to create a server session that maintained state for each client which meant the client didn't have to maintain state,  which in my book is a serious plus.

True? Not?

Posted by Mike Fechner on 22-Feb-2016 06:46

If you can be clearer on the "client".

Posted by Mike Fechner on 22-Feb-2016 06:46

If you can be clearer on the "client".

Posted by Tim Kuehn on 22-Feb-2016 06:52

"client" = smart device - tablet, phone or any other non-PASOE code that talks to PASOE and needs state to do its work.

Posted by Mike Fechner on 22-Feb-2016 06:55

Bound sessions are not supported for REST clients.

Posted by bronco on 22-Feb-2016 07:02

I guess Tim refers to HTTP sessions. I didn't dig into the details but the PASOE admin manual states:

"HTTP sessions are a standard method for maintaining context during a series of requests/responses

between an HTTP client and a Web application."

Posted by bronco on 22-Feb-2016 07:02

I guess Tim refers to HTTP sessions. I didn't dig into the details but the PASOE admin manual states:

"HTTP sessions are a standard method for maintaining context during a series of requests/responses

between an HTTP client and a Web application."

Posted by Mike Fechner on 22-Feb-2016 07:06

http Session work with REST clients as long as they use form based auth. But that won't persist the state on the AppServer. That just keeps a session ID.

Posted by Mike Fechner on 22-Feb-2016 07:06

http Session work with REST clients as long as they use form based auth. But that won't persist the state on the AppServer. That just keeps a session ID.

Posted by Bill Wood on 22-Feb-2016 07:07

WRT:

>>I was under the impression that PASOE enabled one to create a server session that maintained state
>>for each client which meant the client didn't have to maintain state,  which in my book is a serious plus.

>> True? Not?

Answer:

TRUE.  (This is one big advantage of PAS for OE -- you can have more 'sessions'.   The server can process a fixed number of concurrent requests and attach to saved 'state' in the sessions.   This makes programming for STATE-AWARE style work much more feasible on PAS for OE than Classic, but is not an arguement for 'pure performance'. 

Posted by carl.williams on 22-Feb-2016 07:21

Thanks Bill for the clarification. So are we saying that when a machine is under pressure due to maxing out the processors or memory that we see the biggest benefits as there would be more memory/processor free to handle ABL requests as less overall CPU / memory is required to handle the same level of requests. On a system with plenty of "free" resources such as CPU/memory then the benefit will be much smaller.

Posted by Tim Kuehn on 22-Feb-2016 07:36

[quote user="Bill Wood"]

TRUE.  (This is one big advantage of PAS for OE -- you can have more 'sessions'.   The server can process a fixed number of concurrent requests and attach to saved 'state' in the sessions.   This makes programming for STATE-AWARE style work much more feasible on PAS for OE than Classic, but is not an arguement for 'pure performance'. [/quote]

Would this be part of the performance win - not having to transfer or reload state on every call before doing the call's work?

Posted by Irfan on 22-Feb-2016 08:13

For your first statement

             My intention was just to point out the benefit of running one session-manager and agent for various transports, whereas in Classic Appserver you have to run different brokers for each transport.

             The tests were run on each transport individually, not together. A mixed transport environment will definitely show benefit in PASOE

Posted by carl.williams on 22-Feb-2016 08:36

Thanks Irfan

Posted by Peter Judge on 22-Feb-2016 09:32

Having a SESSION-MANAGED client (note the name differs from 'state-aware' or any of the other Classic AppServer mode names) is only possible from an ABL client (APSV transport).
 
The REST, SOAP and WEB transports currently only support SESSION-FREE clients and so there is no session stickiness.
 
 
 

Posted by Bill Wood on 22-Feb-2016 09:40

WRT:

 >> Would this be part of the performance win - not having to transfer or reload state on every call before doing the call's work? 

PAS for OpenEdge does make state-aware much for feasible, but by itself does not affect performance.  You can still use state-aware on Classic AppServer... but the architecture that limits you to locking a session to a request processor (i.e. a Classic Agent) makes this impractical.

Posted by Blake Stanford on 22-Feb-2016 09:45

Clarification.....From what I read in the PAS for OE documentation (APSV transport), state is determined by means of binding the AppServer session, not by Operating Modes (PAS Session-Managed <> Classic "state-aware" when unbound).  Am I understanding that correctly?

Posted by Peter Judge on 22-Feb-2016 09:50

Correct. In PASOE the server can support session-managed and session-free simulataneously.  The client connection determines which is used; in ABL that's via the CONNECT('-URL ... –sessionModel session-managed') statement.
 

This thread is closed