Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added Elastic Renderer and necessary arguments. #563

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Added Elastic Renderer and necessary arguments. #563

wants to merge 1 commit into from

Conversation

maurermj08
Copy link

Added an Elastic renderer. This renderer uses the optional user specified Elastic Search URL (elastic-url) and index name (index). It requires Elastic Search python client, "pip install elasticsearch". I believe this code would be useful for others when threat hunting a large number of RAM images.

Example: "vol.py --profile WinXPSP3x86 -f memory.img --output=elastic --elastic-url="http://192.168.10.100:9200" --index myindex timeliner"

@mutedmouse
Copy link
Contributor

mutedmouse commented Nov 6, 2018 via email

@maurermj08
Copy link
Author

I understand not wanting to bloat volatility core. I believe this renderer would ideally be under the volatility community, but it is my understanding that is not an option at this time. I originally tried going straight from the JSON output to Elastic Search, but the column/row style of the JSON output does not match the format expected by Elastic Search.

This renderer appears to be the simplest and fastest method of exporting to Elastic Search. There are multiple options including writing a custom script that takes in SQLite or JSON, Filebeats, or Logstash. However, these require configuration or writing the data to disk. I remember similar discussions being had about Log2timeline's output module. I originally created a logstash parser. However, others suggested using an output module, which I ultimately found more ideal.

@gleeda gleeda added the renderer label Dec 6, 2018
@pastrom
Copy link

pastrom commented Mar 8, 2019 •
edited
Loading

I really like the idea of adding this as a direct rendrer/output @maurermj08 !

I've earlier been thinking of doing such integration with Elastic myself. Although I might have chosen the easier way of inputting sqlite or json through logstash or filebeat. But I do agree with the arguments you have of doing the direct integration to the elastic-node.

I tried to run some Volatility modules and output them using your script above. Although I got some errors from Elastic. I've tried it with ElasticSearch 5.5 and 6.5. What version of ElasticSearch have you used writing the rendrer?

@maurermj08
Copy link
Author

I did most of my testing on ElasticSearch/Kibana 6.4.2. It uses the official Elasticsearch Python package, https://pypi.org/project/elasticsearch/, so it is easy to switch the version. It should work out of the box on version 6.x. To use a different version of Elasticsearch simply switch the Elasticsearch package to match the appropriate version using PIP:

# Elasticsearch 6.x
elasticsearch>=6.0.0,<7.0.0

# Elasticsearch 5.x
elasticsearch>=5.0.0,<6.0.0

# Elasticsearch 2.x
elasticsearch>=2.0.0,<3.0.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
renderer
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@maurermj08 @mutedmouse @pastrom @gleeda @michael-dolosdev