-
Notifications
You must be signed in to change notification settings - Fork 286
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add RFC5424 (IETF) syslog format handling #149
Comments
this may be more complicated than it at first looks... lots of grok hackery |
@philhagen I have received another message from Synology, which clarifies the <136> integer at the beginning of the IETF message. this is their response:
If i understand correctly it is part of a 'sub' RFC, so they have the RFC police on their side :P i guess it's something to take into account when parsing the IETF messages. |
This is still something that would likely need to be raised with the Logstash developers. I'm not likely going to be able to maintain a processing pipeline that is forked from their codebase. SOF-ELK relies on the Logstash syslog processor. I'll see if there is anything I can do within our scope, but it does not look promising. While they are correct that this technically does adhere to an RFC definition, it is not commonly used - I have never seen a log shipper that uses it. Similarly, HTTP+STARTTLS is defined in an RFC, but I'd be shocked if any browser or server software supports it. |
Okay, thanks for replying. Just to be clear, you are saying to raise this also with the logstash devs? Or should i wait to see if you can check this within the SOF-ELK scope? |
I don't plan to raise the issue, as it's not something I'm directly experiencing. I'll see if there is anything that can be done locally under the context of this issue, but initial research is not promising. |
Are you sure this is not the syslog PRI? A number such as <132> at the
beginning of syslog is usually a mathematically computed syslog PRI.
…On Mon, Mar 25, 2019, 7:42 AM Phil Hagen ***@***.***> wrote:
I don't plan to raise the issue, as it's not something I'm directly
experiencing. I'll see if there is anything that can be done locally under
the context of this issue, but initial research is not promising.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#149 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ALPAz3rlA5gEuNoNxgM2ukdI3O-iGKFvks5vaMRGgaJpZM4bRgiz>
.
|
there are two fields at play - one is definitely the PRI and he totally handle that. The other is a mystery integer that may or may not be the message length. If you look at the screenshot on tagged ticket #144 (https://user-images.githubusercontent.com/4669661/52183368-90c19a80-2807-11e9-8976-0d4d04b334cf.png), you'll see a |
That's a nightmare. Even if it is RFC standard. :)
I don't think Logstash will address this as there are ways to handle it
already albeit ugly ones. The truth is, if this is how Synology is logging,
they need to update how they send logs.
Here's a possible solution with Logstash:
Use aggregate plugin to combine multiple logs back together
Use ruby plugin to add tags identifying if the length has been met
Sincerely,
Justin Henderson
(312) 857-5755
Systems and Security Architect
GSE # 108, Cyber Guardian Red / Blue
http://www.linkedin.com/in/justinhenderson2014/
…On Mon, Mar 25, 2019 at 8:43 AM Phil Hagen ***@***.***> wrote:
there are two fields at play - one is definitely the PRI and he totally
handle that. The other is a mystery integer that may or may not be the
message length. If you look at the screenshot on tagged ticket #144
<#144> (
https://user-images.githubusercontent.com/4669661/52183368-90c19a80-2807-11e9-8976-0d4d04b334cf.png),
you'll see a 136 prefix. The 1 prefix is a "version" field, provided by
the IETF syslog variant. Even the Logstash standard grok syntaxes don't
seem to address the supposed length.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#149 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ALPAz4zZOEEJT9xef8RLbRlf6jMiwMinks5vaNKWgaJpZM4bRgiz>
.
|
LS provides separate RFC5424 grok parsers[1], meaning conditional support is needed for that, in conjunction with necessary field normalization.
[1] https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/linux-syslog
The text was updated successfully, but these errors were encountered: