Raising a QUIT on purpose in a StateFree AppServer session

Posted by slacroixak on 06-Dec-2018 22:49

Under some circumstances ** I'd like a traditional (non PAS-OE yet) StateFree AppServer agent quit its session from the ABL code of its deactivate procedure.

     ** for cleanup purposes, like when this agent finds out some bad code has left some unwanted share-lock records during a hit.

First I tried with a simple QUIT as the doc says the following about the QUIT Statement:

When QUIT is executed from within a procedure running on an AppServer, it terminates the ABL session running on the AppServer, causing the AppServer agent to shut down and returns to the ABL client session from which it was spawned.

But obviously it does not work, as I can still see some record locks from persistent procedures that I started intentionally (for the sake of the test) during the AppServer Hit.

So at the end of the day, I made a work around by firing an OS-COMMAND with asbman -i <AS-name> -stopagent <pid>

I do not like this solution because it may take too long before the asbman -stopagent gets its job done, so the agent may run other requests.

The doc also says the following about PAS-OE:

When QUIT is executed from within a procedure running on a Pacific Application Server for OpenEdge (PAS for OE), the ABL session is reset to its initial state, which includes deletion of persistent procedures and static ABL objects, the disconnection of databases (or re-connection if the databases were connected at startup), and the clean-up of all global data, such as shared variables. Control then returns to the ABL client session from which the server session was accessed, and the server session is returned to its PAS for OE session pool for access by other client requests.

This sounds promising.  But I'd like to be sure this kind cannot be achieved with a Statefree traditional AppServer.  I was told the QUIT does what I wish for the stateless operating mode (it would even quit the agent process), but I am at statefree.

Any comment?

/S

All Replies

Posted by Stefan Drissen on 06-Dec-2018 22:53

function reallyQuit returns logical ():

  reallyQuit().

end function.

Posted by slacroixak on 07-Dec-2018 13:02

What do you "really" mean Stefan ?  Anything logical that I missed ?

Posted by Stefan Drissen on 07-Dec-2018 13:06

The recursive function call will blow up the stack and kill the agent.

Posted by jankeir on 07-Dec-2018 13:13

Nice one Stefan! Never let the ABL tell you you cannot do something ;-)

I have experimented in the past with calling the exit() function from libc for the same use case but never trusted I could safely call it with db's connected. This one happens from time to time due to developer error so it's probably relatively safe (other than maybe leaving some temp files behind and causing a short cpu spike.)

Posted by jankeir on 07-Dec-2018 13:13

Nice one Stefan! Never let the ABL tell you you cannot do something ;-)

I have experimented in the past with calling the exit() function from libc for the same use case but never trusted I could safely call it with db's connected. This one happens from time to time due to developer error so it's probably relatively safe (other than maybe leaving some temp files behind and causing a short cpu spike.)

Posted by slacroixak on 07-Dec-2018 17:09

Hi Stefan, thanks you for your input.  I just gave it a try.  This method sound amazing but cause two problems:

 1) it inserts about 1500 lines in the agent log file to show the terrible offending callstack .  

 2) It raises Progress.Lang.SysError on the client side saying "Server Terminated Unexpectedly".  Perhaps I could manage to catch it an decide to ignore it, but if I get this kind of unexpected error for another reason, then I will miss it :(

doing a hara-kiri with an OS-COMMAND kill <yourOwnPid> leads to about the same result, but without the issue in the log

I wish there would be a way to achieve a nice exit quit, an move the agent outside of the pool so it cannot process any other request.

Any opinion from PSC people?

Posted by Brian K. Maher on 07-Dec-2018 17:14

Why not just do an OS-COMMAND VALUE(“asbman -name blah -agentstop “ + whatever to get pid of process).
 
 
Brian Maher
Principal Engineer, Technical Support
Progress
Progress
14 Oak Park | Bedford, MA 01730 | USA
phone
+1 781 280 3075
 
 
Twitter
Facebook
LinkedIn
Google+
 
 

Posted by Stefan Drissen on 07-Dec-2018 17:25

1. not sure what you have enabled to get a 1500 line callstack, I just get a clean

[18/12/07@18:20:45.138+0100] P-028300 T-017928 1 AS -- Exact Financials 7.25.03 - AppServer started. 11.7.1.0.1429

[18/12/07@18:20:45.138+0100] P-028300 T-017928 1 AS -- ** ABL Debug-Alert Stack Trace **

[18/12/07@18:20:45.138+0100] P-028300 T-017928 1 AS -- --> system/start/appstart.p at line 12661  (..\..\..\build\oe11.7\bl\progress\system\start\appstart.r)

[18/12/07@18:20:52.836+0100] P-022220 T-027132 1 AS -- (Procedure: 'terminateAgent finance/ifallgen/gmcomp.p' Line:14431) HandleAction: bye bye, terminating agent.

The message is given by the function on the /first/ call.

2. a quit would also have triggered something that your client would need to deal with.

Posted by slacroixak on 08-Dec-2018 08:27

Hi Brian, I know my very first post is a bit long.  It was already saying my current work around is an OS-COMMAND asbman -i <AsName> -stopagent <hisPid>   ;)

I have it running in production, and for one case my agent has run 2757 more hits before the asbman -agentstop completed its job :(  (actually, it even fired that asbman command 2757 times...   I've just changed the code with a new global share var to limit to one single call).  So 2757 opportunities to cause a problem.

Brian, I believe we need a may to make an agent tell its broker "Huston I have a problem, please remove me from the agent pool NOW and shut me down, as I must no longer process any more request"

I know the PASOE offers many nice REST management API's.  Is it possible to fire such a remove me from the pool synchronously?  I mean, for some points, the guy that should trigger this action is the agent itself.

Posted by slacroixak on 08-Dec-2018 08:31

Stefan, thank you again for your post.  

1. It may depend on my logmanager setting.  I have the 4GLMessage:4 so an unexpected error shows in the log with its call stack, which may explain that.

2. I agree with you for point.  For now, I believe we might need something new, as said in my post to Brian.

Posted by jankeir on 10-Dec-2018 08:27

There is this:

PROCEDURE exit EXTERNAL '/usr/lib/libc.a(shr_64.o)' CDECL PERSISTENT.

 def input param iexit as long no-undo.

END PROCEDURE.

run exit(0).

You will want to first stop all persistent procedures and disconnect all db's. I do not know though what happens if you do this and you have static classes in memory with that refer to db's connected with shared memory connections.

Posted by jankeir on 10-Dec-2018 08:32

'/usr/lib/libc.a(shr_64.o)' is probably the AIX path, on centos/redhat 7 it's '/lib64/libc.so.6'  I believe.

Posted by slacroixak on 10-Dec-2018 12:41

Jankeir, so far, we target mainly Windows.  That being said, even if I find a way to fire an exit(0) on Windows, I fear it may send a Progress.Lang.SysError on the client side saying "Server Terminated Unexpectedly".  Obviously, the QUIT statement was designed to exit gracefully for some operating mode, and not for others.  I also wanted to point out the need to trigger a graceful exit from the ApServer deactivate procedure.

Posted by jankeir on 10-Dec-2018 14:17

You have my vote if you enter a feature request .

Posted by PhilF on 12-Dec-2018 20:10

Would it make sense to use a FINALLY clause to ensure that resources (such as persistent procedures) are properly cleaned out?

Posted by PhilF on 12-Dec-2018 20:22

(Frankly, I feel like this is so simple-minded that I must be completely missing the true nature of the problem.   So I apologize in advance for my ignorance and/or stupidity.)

> I have it running in production, and for one case my agent has run 2757 more hits before the asbman -agentstop completed its job

I don't work much in this environment -- but isn't it true that that the agent can't be hit again till the existing call exits?  If so, you could just   "repeat:  pause 3000. end." after calling the OS-COMMAND -- waiting till the OS just kills the process.

Posted by PhilF on 12-Dec-2018 20:22

(Frankly, I feel like this is so simple-minded that I must be completely missing the true nature of the problem.   So I apologize in advance for my ignorance and/or stupidity.)

> I have it running in production, and for one case my agent has run 2757 more hits before the asbman -agentstop completed its job

I don't work much in this environment -- but isn't it true that that the agent can't be hit again till the existing call exits?  If so, you could just   "repeat:  pause 3000. end." after calling the OS-COMMAND -- waiting till the OS just kills the process.

Posted by PhilF on 12-Dec-2018 20:23

(Frankly, I feel like this is so simple-minded that I must be completely missing the true nature of the problem.   So I apologize in advance for my ignorance and/or stupidity.)

> I have it running in production, and for one case my agent has run 2757 more hits before the asbman -agentstop completed its job

I don't work much in this environment -- but isn't it true that that the agent can't be hit again till the existing call exits?  If so, you could just   "repeat:  pause 3000. end." after calling the OS-COMMAND -- waiting till the OS just kills the process.

Posted by Stefan Drissen on 13-Dec-2018 07:31

[quote user="PhilF"]

Would it make sense to use a FINALLY clause to ensure that resources (such as persistent procedures) are properly cleaned out?

[/quote]

The FINALLY block is not executed when a STOP condition is raised. A STOP condition is raised by CANCEL-REQUESTS() (sent from client to asynch appserver) or by a LOCK WAIT TIMEOUT (defaults to 10 seconds for WebSpeed agents) .

In 11.7 the -catchStop startup parameter was added which partially numbs the pain - but even with this parameter enabled the FINALLY block is only executed if there is a CATCH Progress.Lang.Stop. 

So FINALLY becomes a diluted 'if you're lucky and have met all the preconditions, we may finally' block.

Posted by Laura Stern on 13-Dec-2018 13:55

Not true.  As of 11.7 with -catchStop (in 12 it will be the default), FINALLY is always executed when there is a STOP condition.  It has nothing to do with having a CATCH block.  Where did you get this notion?

Posted by Laura Stern on 13-Dec-2018 14:05

Sorry - I should clarify, that although -catchStop was introduced in 11.7, getting FINALLY blocks to run after a STOP was only introduced in 11.7.4.  Before that it just did not run, period.  It was never related to having a CATCH block, as is also true for error conditions.

Posted by Laura Stern on 13-Dec-2018 18:19

And to be clearer again, it needs to be -catchStop 1.

Posted by Stefan Drissen on 13-Dec-2018 21:10

The following code with 11.7.1 with -catchStop 1 will /only/ execute the finally block if the Progress.Lang.Stop is caught. Revert the catch to a "normal" Error or even SysError catch and the finally block does not execute. Looking back I now see that catchStop is in the 11.7.3 release notes, but also in the what's new in 11.7 - so I'm lost as to what was introduced when.

I can confirm that in 11.7.4 it finally (sic) works as it should have when finally was originally introduced ten years  ago in 10.1C. 

Glad that this is improved in 11.7.4 and even better that this is the default in 12. 

block-level on error undo, throw.

define button btstop .

define frame fr
  btstop
  .
                          
view frame fr.  
enable all with frame fr.
  
ON 'choose' OF btstop DO:
   stop.
END.

run foo.


procedure foo private: 
   wait-for close of this-procedure.

   catch e3 as Progress.Lang.Stop:   
      message 'caught' view-as alert-box.   
   end catch. 
   
   finally:     
      message 'finally' view-as alert-box.   
   end finally.

end procedure.

Posted by Laura Stern on 13-Dec-2018 21:29

I realize now:  Before 11.7.4, the reason the FINALLY ran when there is a CATCH block for Stop is that the handling of the Stop condition via the CATCH block clears the STOP condition.  Therefore, since there is no longer a STOP condition, the FINALLY will run.  So in effect, you were correct.  However, there is nothing in the AVM that special cases the running of FINALLY because there is a CATCH block for Stop. That's what I was thinking about.  Sorry for the confusion :-0

And glad you like the changes  :-)

Posted by slacroixak on 15-Dec-2018 22:47

Thank you for your posts Laura.  A few days after starting this discussion, we were realizing that our FINALLY blocks do not execute in case of a STOP condition that we even raise ourself on purpose in old pieces of biz logic that cannot yet throw an exception (well that's because of the callers would not be able to catch it yet, so we decided to not adopt the BLOCK LEVEL ON ERROR UNDO, THROW directive for those parts...).  We now suppose this behaviour might cause share lock leaks with dynamic buffers that were supposed to be deleted in these FINALLY blocks.

I am very glad the new behaviour of OE is to finally let these FINALLY blocks execute even (and especially) when a STOP is raised.  Too sad we cannot upgrade to 11.7.4 for now...  (stuck with 11.6.2 or 11.6.4, on too many sites)

This STOP-ignores-FINALLY is probably the reason why I was trying to make my StateFree AppServer Agents QUIT gracefully.  The situation happens rarely, but it can happen.  I have found a fair way to clean up the offending dyn buffers, or possible persistent db-bound procedures (not seen on live yet... by the way it is a bit harder to handle db-bound classes like procedures because they do not offer a property like the procedure DB-REFERENCES attribute...) to avoid a QUIT, but quitting would be much easier and cheaper for such rare cases.

Besides I believe it is a legitimate pattern to detect an agent is in a state that deserves a QUIT. Please le me know if you could suggest a way to achieve such a graceful QUIT.

This thread is closed