Unification of PASOE MS agent logs

Posted by dbeavon on 06-Nov-2018 18:03

Is there any well-known strategy for unifying PASOE MS agent logs?  The ones we are most interested in are the ones ending in <whatever>.agent.log (ie. ABL logs).

(See https://knowledgebase.progress.com/articles/Article/Overview-of-logging-capabilities-in-PAS-for-OE  )

We have lots of ABL apps on a couple different PASOE servers (servers that are load-balanced).  If an error comes up unexpectedly, it is sometimes difficult to know which ABL app to investigate, or even which PASOE server might be hosting that app for a given user at a given moment.

Today we basically have to open lots and lots of logs to find a problem.  It would be especially nice if there was an integration to send errors into the Windows event log, where the errors could be extracted via SCOM (Microsoft System Center).

I  would also be eager to hear about any another option that served the same purpose.  I'm looking for something that is free/cheap and easy to integrate into PASOE.  I want to avoid writing a lot of custom code (reinventing the wheel).  Here are some tools that I've heard mentioned in conjunction with PASOE but I don't know if any are free/easily integrated.

 

  • Elastic Logstash
  • fluentd unified logging
  • New Relic APM

I would be very eager to hear from anyone who may have already solved this problem.

Thanks, David

All Replies

Posted by jankeir on 07-Nov-2018 07:56

Hello,

We are using fluentd to push to elasticsearch and visualize with kibana. You could also use fluentd to push to other things (files, google stackdriver, bigquery,...). It's free and it's easy to set up, configuration took me some time to figure out, but it's now available for you in the docs of this plugin (which you will need to merge multiline messages into one): github.com/.../fluent-plugin-mergecommon

Fluentd is free, local installations of Elasticsearch & Kibana are free as far as storage & aggregation, search and visualisation and  such are concerned (except for your own maintenance and hardware costs off course), there are paid add-ons for alerting and many other things.

I don't know if there is a way to push to the event log, you can however pull from it and push all even logs to kibana so you only have a single place to look at if that's what you're after. If you have used kibana for even a few minutes you will wonder why you wanted to use the windows event log ;-) The insights a search tool that capable and user friendly gives are very useful.

Success,

Jan

Posted by gus bjorklund on 07-Nov-2018 20:12

I have no idea what you are talking about. Too many words I never heard before.

Posted by jankeir on 08-Nov-2018 11:46

Basically grep and tail become a bit cumbersome if you have to do it across hundreds of servers, especially if those servers are being replaced all the time and you want access to the logfiles of servers that are no more.

You might think: then why do you replace the servers all the time? Well: you have to. For one, every time a security patch is released for a server: how do you know the old server was not compromised before you had the chance to apply the patch and has a backdoor? Has the patch really been applied? Or has the attacker replaced apt-get/yum/... Looking at the logs and such? How do you know those are not compromised? ...

Posted by dbeavon on 08-Nov-2018 14:24

Thanks for the pointers Jan.  It sounds like whatever approach I take will involve a bit of a learning curve.  Since the windows event logs are fairly well-established, I was hoping there might be a way to integrate/push the MS-agent log messages into the windows event logs.  (Progress does allow oepas instances to run as windows services... so the next natural step in that direction would be to enable those instances to report their critical problems into an event log in a manner that windows admins would expect).

Posted by bronco on 08-Nov-2018 15:33

I think a lot of us would rather see efforts towards a solution which would work for all OS instead of just Windows. Given the variety of requirements (of customers) I would be surprised to see anything in this respect at all from PSC. Jan's approach sounds a bit more "enterprise" than Windows event log imho.

Posted by jankeir on 08-Nov-2018 16:48

For what it's worth, I'm pretty sure it would not be all that difficult (trivial if you already know ruby) to write a fluentd plugin that would push all events to the windows event log if you really wanted that.

Essentially it's combining this: rubygems.org/.../0.6.5 with docs.fluentd.org/.../plugin-development

But personally I think it would be impractical for anything close to a significant amount of logging, I'm not too fond of the windows event log, then again, you may have guessed that I am more of linux person.

This thread is closed