Fluent Patterns for OE logs

Posted by jbeisch on 26-Mar-2019 19:46

Wondering if there's anyone that has done the legwork of creating fluentd/bit patterns for OE logs such as the database, CASOE and PASOE processes? Thanks!

All Replies

Posted by Peter Judge on 26-Mar-2019 19:56

I think you’ll be able to get a config in the PASOE Docker container (on ESD?). The PASOE agent  log format is very similar to CAS.

Posted by jankeir on 27-Mar-2019 08:10

Database for fluentd:


 @type tail

 format /^\[(?<time>\d{4}\/\d{2}\/\d{2}@\d{2}:\d{2}:\d{2}\.\d{3}\+\d{4})\] P-(?<pid>\d+)\s+T-(?<thread>\d+)\s+(?<severity>\S) (?<connection.type>(SQLSRV2)|([a-zA-Z]+))(\s*)(?<connection.id>[^:\s]+): .(?<messagenumber>[\d-]+).(\s+)(?<message>.*)$/

 path /some/db.lg

 pos_file /var/log/td-agent/somedb.lg.pos

 read_from_head true

 tag somedb


Clientlog for fluentd:


 @type tail

 format /^\[(?<time>\d{2}/\d{2}/\d{2}@\d{2}:\d{2}:\d{2}\.\d{3}\+\d{4})\] P-(?<pid>\d+) T-(?<thread>\d+) (?<loglevel>\d+) (?<executionenvironment>[^\s\\]+) (?<logentrytype>[^\s\\]+)\s+(?<message>.*)$/

 path /some/client.log

 pos_file /var/log/td-agent/someclient.log.pos

 read_from_head true

 tag someclient

 path_key logfile


For pasoe logfile processing with fluentd and for clientlog parsing you may be interested in this, which handles multiline messages:


[mention:9e4ee96fac634b8f91b580e1fb4f7e71:e9ed411860ed4f2ba0265705b8793d05] I haven't looked at the parsing provided by progress recently, does it now handle multi-line messages? I wouldn't mind switching to fluent-bit if it's supported now.

Posted by jbeisch on 29-Mar-2019 15:28

Thanks guys, great help from you both!

This thread is closed