Computing a variable from a column and filtering the column by it

[zikook]

Hi,
I have columns which need to be filtered by removing variables less than the mean + standard deviation (calculating the mean+stdev individually for each column). This can't be a static variable since I need a flow where every few days a new csv file is loaded.
(I'm aware that after filtering, the columns might not be the same length - that is logical to our data and addressed elsewhere in our flow).
Is there a way to calculate a cutoff variable and use it as input for a filter?
A simple example- a column with values [1,2,3,40,100], where mean+stdev = 72.0684. Removing entries below this value would only leave 100.
Thanks!
-Aharon

[zikooky]

Hi,
in case anyone is interested, I found a simple solution using the awesome python script widget.

#v stores the column position of the column we're interested in
v = in_data.domain.index("glucose1")
m = np.mean(in_data[:,v])
std = np.std(in_data[:,v])
#only output rows whose glucose values are greater than mean + standard deviation
out_data = Table.from_list(in_data.domain,[d for d in in_data if d["glucose1"] > (m+std)])

-Aharon

[wvdvegte]

Yes, I think you can only do this using Python Script.
Having the possibility to calculate a variable (like the standard deviation, which can be extracted from your data with Feature Satistics), and feeding it to the Formula widget together with the data set to which it is applied in a calculation would be a nice-to-have in Orange.
RapidMiner, which is a commercial software package with functionality similar to Orange, offers this possibility in the form of so-called macros (which I think is a confusing name)...