Home
Welcome to the DAD wiki!
Two important items:
- If you have been updating, you will have noticed that searches became incredibly slow while inserts become very fast. The root cause was that the search was relying on a table join against words even though it was no longer necessary. Since there was no longer an index between the two, this resulted in sub-second queries taking 15 or 20 minutes. This has been resolved.
- The events are being inserted as raw text into the Events table. Currently, all single quotes are being changed to double quotes because I haven't taken the time to turn this into a bound query. Normally this would be the obvious and simple solution, but since the inserts are batched into several thousand, it is not nearly as simple to solve. Eventually this will be addressed as a bug fix and single quotes will be permitted in the database again. For now be aware that single quotes have been doubled.
A major architectural change has occurred. We are no longer tracking word positions in events. The decision to store hunks has also been struck down based on metrics. We are now capable of handling more than 36,000,000 events per day after a great deal of time with ruby_prof and some strategic threading. There may be even more opportunities for improvement.
In the meantime, there will definitely be some oddities in event searching and display. The good news is that we are now storing 100% fidelity of the event. The bad news is that this will likely require you to rewrite display filters and jobs since events may now have mixed case and additional characters that would have been removed previously.
DAD has undergone some really substantial changes in the last few months. It went from being an absolutely mess of PHP, Java, Perl and JavaScript to being almost entirely Ruby on Rails. Of course, there's still some JavaScript in there and I have not yet gotten around to rewriting the syslog listener as a Ruby script, but that's coming soon. As of April, 2016, DAD is able to hit sustained insert rates of more than 400 events per second on a small 16 gig development server.
More than anything else I decided to get this Wiki running to start documenting useful jobs that DAD is great for searching for and alerting on. Bear in mind that the software is absolutely, unequivocally, Alpha software at the moment. For example, even though I have the entire rights and user management system sitting in a directory, I have violated my own standards for getting that incorporated first in favor of some fast functionality. What this means is that DAD is likely rife with vulnerabilities, and I know it. Frankly, you have the ability to jam Ruby code right into it as an automated job so I'm not too worried just yet. :)
Anyway, what you're likely most interested in for the moment are example jobs: