Unique ID for PASOE ABL session?

Posted by ssouthwe on 30-Jul-2019 16:41

Is there any way within an ABL session under PASOE to get some the Process ID (PID) of the agent, and some sort of unique identifier of the specific session within the multi-session agent?

In the appname.agent.log file, the prefix for each entry appears to show PIDs, as well as some unique values preceded by T- and AS-:

[19/07/30@11:22:02.119-0500] P-015080 T-008024 1 AS-8 ...

I'm wondering how I could get those programmatically, since it would be useful for troubleshooting.

Thanks in advance.

All Replies

Posted by Irfan on 30-Jul-2019 17:44

Hi,

In 12.0 we have added a feature called as RequestID which is the unique identifier of a request in PASOE. It will be in the format of <WebApp>:<transport>:<RequestID>.

Example:- ROOT:r:00001

It will be generated in all the logs in PASOE and if you want to get it in the code, then you can get it from the RequestInfo Object.

Posted by Peter Judge on 30-Jul-2019 18:02
Take a look at the OREquestInfo object, which is available as an attribute on the SESSION handle. That has some of what you're after. Not the PID though.
 
Posted by ssouthwe on 30-Jul-2019 18:09

Irfan, if you're talking about CURRENT-REQUEST-INFO:RequestID, that's present in my 11.7.4 product, but the ID at the end keeps incrementing on each request, and doesn't match the agent log.  

SESSION:CURRENT-REQUEST-INFO:RequestId = Appname:w:00000024

I'm actually looking for something that identifies which particular session of the multi-session agent.

For example, in WebSpeed, you can get the PID by doing this:

ASSIGN cAgentPID = ENTRY(3,WEB-CONTEXT:EXCLUSIVE-ID,":").

Now that we can have more than one session per OS process, I need to track specifically which session has done something.

Posted by Irfan on 30-Jul-2019 18:39

In your <PASOE Instance>/bin/openedge_setenv.sh, you should find the following line.

#REQIDLOGGING=y ; export REQIDLOGGING

Enable this line, to get the RequestID in the agent log.

Posted by ssouthwe on 30-Jul-2019 19:20

Irfan, thanks but I'm not looking to put the requestID in the log.  I'm trying to get the PID and Thread ID.

I've found that this code works (as an embedded Speedscript file) but it's not ideal to have to write and read just to get these.

<!DOCTYPE=html>
<script language="speedscript">
DEFINE VARIABLE cTextIn AS CHARACTER NO-UNDO.
DEFINE VARIABLE cPID AS CHARACTER NO-UNDO.
DEFINE VARIABLE cAgentLog AS CHARACTER INIT "C:\apps\lisa\logs\ingridabl.agent.log" NO-UNDO.
DEFINE VARIABLE cThreadID AS CHARACTER NO-UNDO.
DEFINE VARIABLE cUniqLogString AS CHARACTER INIT "_PID_THREADID_GET_&1_" NO-UNDO.

cUniqLogString = SUBSTITUTE(cUniqLogString,STRING(RANDOM(1,1000000))).    
FILE-INFO:FILE-NAME = cAgentLog.   
INPUT FROM VALUE(cAgentLog).
SEEK INPUT TO FILE-INFO:FILE-SIZE.
MESSAGE cUniqLogString. /* Stick it in the log file */
REPEAT ON ENDKEY UNDO, LEAVE:
	IMPORT UNFORMATTED cTextIn.
	IF cTextIn MATCHES "*" + cUniqLogString + "*" 
	    THEN ASSIGN
	    cPID = SUBSTRING(ENTRY(2,cTextIn," "),3)
	    cThreadID = SUBSTRING(ENTRY(3,cTextIn," "),3)
	    NO-ERROR.
END. /* repeat */
INPUT CLOSE.    
    
</script>
<html> 
<body>
<div class="bodycontent">
<h1>Agent Info</h1>
<p>Information about the agent</p>
`cTextIn`
<table>
    <tr>
        <td>Process ID</td> 
        <td>`cPID`</td>
    </tr>
    <tr>
        <td>Thread ID</td> 
        <td>`cThreadID`</td>
    </tr>
</table>
</div>
</body>
</html> 
   

 

Posted by Irfan on 30-Jul-2019 19:52

Why not use RequestID as a unique identifier to debug the issue. You will get this information in all logs(tomcat and MS-Agent). Why do you need to track the agentId or parse the agent log?

Posted by ssouthwe on 30-Jul-2019 20:03

In WebSpeed, sometimes you would have an agent get "stuck" and you could see that it was just burning resources and not completing the request.  When a user gets impatient, they sometimes repeat the same request and get all your agents stuck.

In the past, we'd see a given PID that had gone sideways, and need to figure out what it was running and who was running it.  If the application code can figure out its PID (and in this case, it's thread) then it can be made to store that info along with other details about the request as a way of troubleshooting.  So when something freezes in PID 999, you scan the records and see what the last thing was that started to run on that PID, and who the user was.

Another thing we would do is have inter-process communication between WebSpeed agents for various things like having them refresh cached data after an update.  With PIDs and ThreadIDs, we could track which agents have picked up the communication and performed the requested action.

My hope is to have all the same abilities that we relied on in WebSpeed available in PASOE for these odd use cases.

Posted by Irfan on 30-Jul-2019 20:13

For monitoring and understanding which requests are stuck and what each request is running, we have added API's like oemanager and JMX. There are more than 50+ API's that tells you what is happening inside the server. We even have API's that tells us the which ABL Procedures were executed, which requests are hung by providing a timeout value and the stacks of all the sessions in the agent and individual ABL session.

But if you would like to use old style, then you can still use RequestInfo Object. You can use session:current-request-info:AgentId  to get the AgentID and similary, there should also be SessionID attribute to get the ABL session value.

So we have the data, but I would incline towards using API's in an automated way. In-case you need specific API's which are not available then please let us know. We can work on providing API's that will be useful for you to debug issues with PASOE in runtime.

Posted by ssouthwe on 30-Jul-2019 22:03

Thanks for the reply. I've spent a little time looking at oemanager, but the lack of documentation is slowing me down a bit.  Unformatted json is also making it hard to read.  

If I'm looking at the right document, it really doesn't describe what all the fields are.  For example, this page shows example output, but doesn't describe what SessionState is, or what all the possible values mean.

It also doesn't seem like it has an API to show requests, unless you know what agent you want to look at.


Let me back up a bit and ask this:

What do AgentID, SessionID and ThreadID mean in the context of this?

When I asked for a ThreadID, what I was looking for was something unique and permanent to the individual ABL session within an agent.  From what I can tell, the ThreadID can change from request to request against the same ABL session.

Posted by gus bjorklund on 30-Jul-2019 23:22

to get the process id on linux, set up link to system call:

procedure getpid external "/usr/lib/glibc.so" cdecl:

define return parameter pid as long no-undo.

end procedure.

/* then to use it: */

def var p as integer no-undo.

p = getpid ().

on windoze:

procedure GetCurrentProcessId external "kernel32.dll":

define return parameter pid as long.

end procedure.

and:

def var p as integer no-undo.

run GetCurrentProcessId (output p).

Posted by dbeavon on 31-Jul-2019 00:10

I didn't see anyone mention it but OEE allows you to browse all the agents of an ABL application, and explore the sessions that are running within each of them.  You can even click to see the callstack for each of the sessions.  If one of them is "stuck" then you will find it in OEE with a related session state (something other than IDLE).

Are you windows or Linux? I find that the oemanager REST api is helpful for management.  You can use it with curl or powershell.  Below is session detail for a certain agent (for some reason agents in oemanager are identified by some strange looking character string, that is not the PID).  see screenshot from an oemanager REST request.

If you want a powershell example, you can get a basic example here (the purpose of this was to trim sessions, but the same structure might apply to investigating misbehaving sessions too):

https://community.progress.com/community_groups/openedge_development/f/19/t/36461

Posted by Irfan on 31-Jul-2019 10:59

We Provide Swagger for OEManager API's and I think it provides ample documentation. Once you know which API's you want to use, you can choose either REST API's or JMX to automate them. David has given an example of using REST API's in this thread. If you would like to do it using JMX, then you can use OEJMX.

Documentation to enable Swagger for OEManager - docs.progress.com/.../Enable-Swagger-UI-for-management-REST-API-access.html

Documentation for oejmx utility - docs.progress.com/.../Use-OEJMX-to-manage-and-monitor-an-instance.html

An ABL Session can have more than 1 threads and so an ABL session can have more than 1 ThreadID's. It is only useful for us to debug the Multi-Session agent. For your context, each ABL session can be tracked by its session-id and agent-id. But if you would like to monitor requests that are getting executed on that MS-Agent, then RequestID would be the key to debug. Please enable Swagger and look at the API's in "AgentManager" section and see if any of them matches your case.

Posted by Jean-Christophe Cardot on 31-Jul-2019 12:30

Hi

In order to get the agent number in PASOE I'm using SESSION:CURRENT-REQUEST-INFO:SessionId

(if I did not miss this, it was not mentionned in this thread).

++

JC

Posted by Peter Judge on 31-Jul-2019 14:08
 
There's an AgentId property on the OERequestInfo object that's no in the doc. SessionId is not the same as AgentId.
 
Posted by ssouthwe on 31-Jul-2019 14:32

Irfan, thanks.  I've enabled Swagger, and I'm exploring that right now.  

It looks like it only provides interfaces to some of the APIs, but not all.  For example /applications/{appname}/agents/{agentID}/sessions that David shows above is missing.

I'd disagree with you on the notion that Swagger provides ample documentation.  The swagger screens provide no documentation on what the fields mean, and only a very brief description of what the API does.  While the names of the various properties and such in the responses may mean something to the engineering team, they are not all self-explanatory.  

I'm not very familiar with Swagger, but looking at other examples out on the web, I'm seeing the ability to put a lot more documentation into it.  For example, this implementation shows a model for the responses that provides information on the fields.

Is there a reference somewhere else for this that explains each API and the output?

Posted by Irfan on 31-Jul-2019 15:14

Thank you for your comments Steve. If you think the API description are not sufficient or they do not have enough information ,then please raise a ticket with Tech Support. We will definitely try to improve in that area.

I have attached the screenshot of the API that David pointed out. Please take a look

[View:/cfs-file/__key/communityserver-discussions-components-files/19/oemanagerAPI.docx:320:240]

Posted by ssouthwe on 31-Jul-2019 15:55

As a followup question, I'm wondering how SessionID and AgentID from SESSION:CURRENT-REQUEST-INFO is related to the oemanager APIs?

For example, testing just now in my app:

SESSION:CURRENT-REQUEST-INFO:AgentID = 4932

SESSION:CURRENT-REQUEST-INFO:SessionID = 7

but the oemanager/applications/{appname}/sessions API returns this, which has AgentID and SessionID inside it, but with random Base64-looking values that don't appear to match up:

{
  "result": {
    "OEABLSession": [
      {
        "adapterType": "",
        "bound": false,
        "requestID": "",
        "elapsedTimeMs": 395921,
        "ablSessionID": "",
        "lastAccessStr": "2019-07-31T10:44:02.014-0500",
        "agentConnInfo": null,
        "sessionID": "lR8JgyIwR6uODfpOhoP4tg",
        "clientConnInfo": null,
        "sessionState": "RESERVED",
        "agentID": "",
        "requestState": "READY",
        "sessionPoolID": "CZDEum6oSD-ylM2TUt91KQ",
        "sessionType": "SESSION_FREE"
      },
      {
        "adapterType": "WEB",
        "bound": false,
        "requestID": "Ingrid:w:0000001a",
        "elapsedTimeMs": 5265,
        "ablSessionID": "",
        "lastAccessStr": "2019-07-31T10:50:32.670-0500",
        "agentConnInfo": {
          "state": "RESERVED",
          "connPoolID": "osovtHUPTCifAUctF8_Q5w",
          "connID": "7LEBvJGISKWrJ2eSQD1KDw",
          "agentAddr": "/0.0.0.0:62002",
          "localAddr": "/127.0.0.1:51745",
          "agentID": "g5Ce8GboT-KMFnGC5vd5Pg"
        },
        "sessionID": "64tl7R6UQnmeuW7rvysSZw",
        "clientConnInfo": {
          "adapterType": "WEB",
          "requestProcedure": "Progress.Web.InternalWebHandler:HandleRequest",
          "requestID": "Ingrid:w:0000001a",
          "elapsedTimeMs": 5274,
          "httpSessionId": "FF1098787A8A28B4BDF91D5E767D22BEB8619D60D1A5.lisa",
          "sessionID": "64tl7R6UQnmeuW7rvysSZw",
          "reqStartTimeStr": "2019-07-31T10:50:32.662-0500",
          "executerThreadId": "thd-5",
          "clientName": "0:0:0:0:0:0:0:1",
          "requestUrl": null
        },
        "sessionState": "RESERVED",
        "agentID": "g5Ce8GboT-KMFnGC5vd5Pg",
        "requestState": "RUNNING",
        "sessionPoolID": "CZDEum6oSD-ylM2TUt91KQ",
        "sessionType": "SESSION_FREE"
      },
      {
        "adapterType": "WEB",
        "bound": false,
        "requestID": "Ingrid:w:0000001b",
        "elapsedTimeMs": 4054,
        "ablSessionID": "",
        "lastAccessStr": "2019-07-31T10:50:33.882-0500",
        "agentConnInfo": {
          "state": "RESERVED",
          "connPoolID": "osovtHUPTCifAUctF8_Q5w",
          "connID": "KYODMP_ZSqauAKK1_-00Lg",
          "agentAddr": "/0.0.0.0:62002",
          "localAddr": "/127.0.0.1:51778",
          "agentID": "g5Ce8GboT-KMFnGC5vd5Pg"
        },
        "sessionID": "iSaU_7cRTFyDN0ldMP8ziQ",
        "clientConnInfo": {
          "adapterType": "WEB",
          "requestProcedure": "Progress.Web.InternalWebHandler:HandleRequest",
          "requestID": "Ingrid:w:0000001b",
          "elapsedTimeMs": 4059,
          "httpSessionId": "FF1098787A8A28B4BDF91D5E767D22BEB8619D60D1A5.lisa",
          "sessionID": "iSaU_7cRTFyDN0ldMP8ziQ",
          "reqStartTimeStr": "2019-07-31T10:50:33.877-0500",
          "executerThreadId": "thd-10",
          "clientName": "0:0:0:0:0:0:0:1",
          "requestUrl": null
        },
        "sessionState": "RESERVED",
        "agentID": "g5Ce8GboT-KMFnGC5vd5Pg",
        "requestState": "RUNNING",
        "sessionPoolID": "CZDEum6oSD-ylM2TUt91KQ",
        "sessionType": "SESSION_FREE"
      }
    ]
  },
  "errmsg": "",
  "outcome": "SUCCESS",
  "versionStr": "v11.7.4 ( 2018-10-10 )",
  "versionNo": 1,
  "operation": "GET CLIENT SESSIONS"
}


Posted by Irfan on 31-Jul-2019 16:06

Steve,

The sessionId's you are looking at from SessionManager which are tomcat sessionId's. You should be looking at the AgentManager API's to get the ABL SessionId's. In the document attached I have provided you the exact API.

When you access "Get Agents" API, you get two values "AgentPID" and "AgentID". AgentPID is the MS-Agent PID and AgentID is the unique identifier of the agent used for administration purposes by not exposing the agent PID information.

Posted by ssouthwe on 31-Jul-2019 16:47

Please pardon my ignorance here - I think the nomenclature is throwing me off.  There are so many things called "session" that it's confusing me.  It's not clear to me in some cases whether session relates to a session in the MS agent, something tomcat related, or the HTTP session of the user.

What I'm really looking for in all of this is how I can use whatever tools are available to remedy a stuck request.

So, if my MS Agent has several sessions going, each of which may be serving a request, but one of the requests is hosed and has become a runaway process, how would I find that and kill it?  How would I take the information gleaned in finding what Agent/Session/Request failed, and match that up to data that my app can store, and use that to hunt down and fix the bug?

Is it even possible to kill just one session in the MS Agent?  You wouldn't want to kill the whole agent and disrupt everyone's work to fix one problem, right?

Edit:  I just figured this out, so I'll share in case it helps someone else:

First, you need the agent PID and the MS Agent SessionID for the abl session running the program.

If for example, the agent PID is 4932 and the sessionID is 7, then do this:

  1. Log into the oemanager

  2. Run /applications/{appName}/agents with the name of the application

  3. Locate the agent PID within the results, and copy the corresponding agentID value

  4. Now with that agentID value, run the /applications/{appName}/agents/{id}/{component}

    1. Use the copied agentID instead of {id}. (In this case, g5Ce8GboT-KMFnGC5vd5Pg)

    2. Use “sessions” instead of {component}

    3. You can verify that the session is still busy with the request, and that it’s the correct one you want to kill.

  5. Run the DELETE on /applications/{appName}/agents/{agentID}/sessions/{sessionID}

    1. Use the copied alphanumeric agentID (In this case, g5Ce8GboT-KMFnGC5vd5Pg)

    2. Use the integer SessionID  (In this case 7)

Posted by Irfan on 31-Jul-2019 18:09

Thank you Steve, I was about to respond the same. There are different ways of doing it. This API basically makes sure the HTTP request is terminated which will make sure the ABL Session that is executing will be cancelled.

In-case you want to terminate an ABL Session, then you can do it by executing "Terminate ABL Session" API using the DELETE operation. In-case you want to see where it is going wrong, then look for "Get MS-Agent stacks" API.  If your agent is in a bad state( growing memory or hosed) you can use "stop MS-Agent" API which will allow the existing requests to finish and the new requests will be routed to a new agent.

For your use-case, first option is the right choice.

This thread is closed