Wondering if there's anyone that has done the legwork of creating fluentd/bit patterns for OE logs such as the database, CASOE and PASOE processes? Thanks!
Database for fluentd:
<source>
@type tail
format /^\[(?<time>\d{4}\/\d{2}\/\d{2}@\d{2}:\d{2}:\d{2}\.\d{3}\+\d{4})\] P-(?<pid>\d+)\s+T-(?<thread>\d+)\s+(?<severity>\S) (?<connection.type>(SQLSRV2)|([a-zA-Z]+))(\s*)(?<connection.id>[^:\s]+): .(?<messagenumber>[\d-]+).(\s+)(?<message>.*)$/
path /some/db.lg
pos_file /var/log/td-agent/somedb.lg.pos
read_from_head true
tag somedb
</source>
Clientlog for fluentd:
<source>
@type tail
format /^\[(?<time>\d{2}/\d{2}/\d{2}@\d{2}:\d{2}:\d{2}\.\d{3}\+\d{4})\] P-(?<pid>\d+) T-(?<thread>\d+) (?<loglevel>\d+) (?<executionenvironment>[^\s\\]+) (?<logentrytype>[^\s\\]+)\s+(?<message>.*)$/
path /some/client.log
pos_file /var/log/td-agent/someclient.log.pos
read_from_head true
tag someclient
path_key logfile
</source>
For pasoe logfile processing with fluentd and for clientlog parsing you may be interested in this, which handles multiline messages:
github.com/.../fluent-plugin-mergecommon
[mention:9e4ee96fac634b8f91b580e1fb4f7e71:e9ed411860ed4f2ba0265705b8793d05] I haven't looked at the parsing provided by progress recently, does it now handle multi-line messages? I wouldn't mind switching to fluent-bit if it's supported now.
Thanks guys, great help from you both!