Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Have flexibility at the column level to define anomalies. 3 new features:
Resolves re-data/re-data#413
How
I am bringing a new model
re_data_selected_columns
that has the config per each column. When a column config is present,re_data_anomalies
will read fromre_data_selected_columns
instead ofre_data_selected
.Example of column anomaly config:
In the case above, for the avg(price) we will consider a strict z_score of 0.1 which overrides the values 3 on the model. Moreover, the values should be higher than 5200 and the percentage of change has to be higher than 5% for this to become an anomaly