Automated outlier detection for adjust sum dialog #18723
Conversation
| addOutlier(c.hour); | ||
| } else { | ||
| c.fiveMin.forEach((s) => { | ||
| addOutlier(s); |
There was a problem hiding this comment.
I don't think you can process 5min and hour the same way, a hour change normally already is 12 times as big?
| }; | ||
|
|
||
| // If an hour has no five minute data, add the hour value | ||
| // Otherwise, add the 5 minute values and ignore the hour value |
There was a problem hiding this comment.
An hour can have only half of the 5 minute data right? So we could miss a bunch?
There was a problem hiding this comment.
I thought about this a bit and I'm not sure if I can really think of a case where there would be a problem here. Possibly a minor edge case during the single hour where 5 minute data has been partially purged? But that just seems really unlikely to be a problem.
I'm not really sure why we're even dealing with 5 minute data here at all as it is just temporary, but I was just sort of trying to mimic how the dialog already handled overlapping 5min/hour data.
|
I think this should really be done on the backend, and we should maybe even raise issues for it if the value is way different than the normal value. |
| const numOutliers = 10; | ||
|
|
||
| // Track the top 10 values. | ||
| const addOutlier = (s) => { |
There was a problem hiding this comment.
I would say an outlier is not the biggest 10 numbers, but should be x% higher or lower than the mean change value?
There was a problem hiding this comment.
I'm not sure the distinction would matter that much for the typical use case here, but I'm not opposed to doing that (it's just more cpu work required to calculate the mean).
There was a problem hiding this comment.
I agree it probably doesn't add much
Proposed change
Add a button in the adjust-sum dialog to automatically search statistics for outliers (the top 10 largest change values in the stat history).
I decided to try this as I find the current process somewhat tedious to find values to change. It is possible that you have an erroneous energy value, but just know what day it occured on, and not know the time. Then it requires scanning through the entire day by manually adjusting the datetime picker for every 20-30 minutes through the entire 24 hour period, which involves having to switch between changing minutes, changing hours, changing am/pm until you find it.
In all cases where I've had to do this it is because some value had such a large change that it would be instantly recognizible just by sorting the statistics by magnitude of change. So this button approximates that process, and allows to locate the bad values quickly with one click.
Type of change
Example configuration
Additional information
Checklist
If user exposed functionality or configuration variables are added/changed: