Release v0.7.0

[jshlbrd]

Hey folks, this is our announcement for the release of Substation v0.7.0! 🎉 We're planning to do these for major and minor releases moving forward to provide a deeper look at what's changed in each release.

New External Enrichment Processors

This release adds two new external enrichment processors -- a DNS processor and IP database processor.

The DNS processor enriches domains and IP addresses by querying DNS and as of this release can do forward lookups (domain to IP addresses), reverse lookups (IP address to domains), and TXT queries (returns TXT records for a domain). This can be combined with the domain processor to do targeted domain querying or used to enrich data from third party services like Team Cymru's Malware Hash Registry. It's easy for us to add support for other DNS query types, so if there's anything you'd like to see, then file an issue and let us know.

The IP database processor enriches IP addresses by retrieving information from enrichment databases such as MaxMind and IP2Location. There are two super cool features in this processor:

• database files are dynamically fetched from either local disk, HTTP(S), or AWS S3 -- as long as the processor / Substation application has access, then you're good to go!
• we can add support for new database providers -- this processor is designed to easily integrate new service providers (assuming they've published a Go package for their service)

We've also added a new libsonnet pattern for using the IP database processor -- this pattern verifies that the input is a valid public IP then applies the processor.

Replace with Nothing

The replace processor can now replace text with nothing -- this is great for cleaning up junky event logs that might insert nonsensical characters into their data!

More IP Inspection

The IP inspector now supports identifying if a value is a valid IP address -- before this release we only had options to test for types of IP addresses, but not if the value is any type of IP address.