Logstash Grok filters for OpenEdge

Posted by danielb on 10-Jul-2018 01:01

We are currently in the process of implementing an ELK stack for log file archival, searching, etc. and the hard(est) thing we had to do was create the GROK filters for Logstash. The format of the log files across the AppServer broker, Server, NameServer and Database are all slightly different.

I'm posting them here, in hopes that they will be useful to other developers that need to achieve the same thing, as well as to get some comments/feedback on them from anyone else that has implemented the same.

OpenEdge Database:

^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_ID:pid}\s*%{OE_THREAD_ID:thread_id}\s*%{WORD:message_type}\s+%{WORD:db_process_type}\s*(%{OE_USER_ID:user_id})?\:\s+\(%{OE_MESSAGE_NUM:progress_message_num}\)\s+%{GREEDYDATA:message_text}

OpenEdge AppServer/Webspeed:

^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_ID:pid}\s*%{OE_THREAD_ID:thread_id}\s*%{BASE10NUM:logging_level}\s+%{NOTSPACE:exec_env}\s*%{NOTSPACE:exec_subsys}\s*%{GREEDYDATA:message_text}\s*\(%{BASE10NUM:progress_message_num}\)$

or


^%{OE_TIMESTAMP:[@metadata][timestamp]}\s*%{OE_PROCESS_ID:pid}\s*%{OE_THREAD_ID:thread_id}\s*%{BASE10NUM:logging_level}\s+%{NOTSPACE:exec_env}\s*%{NOTSPACE:exec_subsys}\s*%{GREEDYDATA:message_text}

We have also defined the following types specifically for the patterns:

OE_PROCESS_ID \P\-[0-9]*
OE_THREAD_ID \T\-[a-zA-Z0-9\-]*
OE_TIMESTAMP %{SYSLOG5424SD}
OE_USER_ID %{BASE16FLOAT}
OE_DB_PROCESS_TYPE %{WORD}
OE_DB_MESSAGE_TYPE [FI]
OE_MESSAGE_NUM \(?%{BASE16FLOAT}\)?

Hopefully someone else finds this information useful.

All Replies

Posted by bronco on 10-Jul-2018 01:33

Thanks! This is very useful since devising these is always a bit of a pain.

This thread is closed