Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

timestamp reordering (should be first) #168

Open
arianf opened this issue May 1, 2022 · 1 comment
Open

timestamp reordering (should be first) #168

arianf opened this issue May 1, 2022 · 1 comment

Comments

@arianf
Copy link

arianf commented May 1, 2022

timestamp should be the first thing in the hash, given that splunk will only parse by default the first 128 characters of a json MAX_TIMESTAMP_LOOKAHEAD default is 128

Since message is almost always a variable length, and timestamp is always a fixed length, I think by default the order of timestamp should be first in the ruby hash.

event = case data
when LogStash::Event
data.clone
when Hash
event_data = data.clone
event_data['message'.freeze] = event_data.delete(:message) if event_data.key?(:message)
event_data['tags'.freeze] = event_data.delete(:tags) if event_data.key?(:tags)
event_data['source'.freeze] = event_data.delete(:source) if event_data.key?(:source)
event_data['type'.freeze] = event_data.delete(:type) if event_data.key?(:type)
event_data['@timestamp'.freeze] = time
LogStash::Event.new(event_data)
else
LogStash::Event.new("message".freeze => msg2str(data), "@timestamp".freeze => time)
end

could be rewritten to:

        event = case data
                  when LogStash::Event
                    data.clone
                  when Hash
                    event_data = data.clone
                    event_data['@timestamp'.freeze] = time
                    event_data['message'.freeze] = event_data.delete(:message) if event_data.key?(:message)
                    event_data['tags'.freeze] = event_data.delete(:tags) if event_data.key?(:tags)
                    event_data['source'.freeze] = event_data.delete(:source) if event_data.key?(:source)
                    event_data['type'.freeze] = event_data.delete(:type) if event_data.key?(:type)
                    LogStash::Event.new(event_data)
                  else
                    LogStash::Event.new("@timestamp".freeze => time, "message".freeze => msg2str(data))
                end
@arianf
Copy link
Author

arianf commented May 1, 2022

In the meantime, I have the following workout which is not great:

customize_event: lambda do |event|
  event.instance_variable_set(
    :@data,
    { timestamp: event.timestamp.iso8601(6) }.merge(event.instance_variable_get(:@data))
  )
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant