Model Update Issues

[reasondk]

"In the river framework, for tree models such as HoeffdingTreeClassifier, are the model parameters completely updated or updated based on the previous ones in each iteration of the fit process?"

[MaxHalford]

Hey there.

I'm sorry I don't understand your question. Could you add some details/context?

[reasondk]

@MaxHalford Thank you for your response. In each iteration of the for loop, every time the model is fitted, it means an update of the parameters. What I want to know is whether each fitting updates all the parameters of the model or if it updates only part of the parameters based on the model generated in the previous iteration of the loop.

from river import tree, metrics
df = pd.DataFrame(np.array([[1, 2, 3, 1],
                            [4, 5, 6, 0],
                            [7, 8, 9, 0],
                            [10, 11, 12, 2]]), columns=['f1', 'f2', 'f3', 'label'])

increase_model = tree.HoeffdingTreeClassifier(max_depth=30)
for index, row in df.iterrows():
    X = row[['f1', 'f2', 'f3']].to_dict()
    y = row['label']
    y_pred_increase = increase_model.predict_one(X)
    y_prob_increase = increase_model.predict_proba_one(X)
    increase_model.learn_one(X, y)   # All parameters update? Or based on the previous iteration, partial parameters update?```

[MaxHalford]

I'm not sure I understand but here goes: the model is stateful, and doesn't start from scratch each time you call learn_one. That's the whole point of online learning.

Does that help?

[reasondk]

Thank you, I understand. I have another question. During the process of incremental learning, I noticed that the model keeps growing, especially when using the HoeffdingTreeClassifier for multi-classification. The size of the model in memory continues to increase, and if this continues, it could lead to OOM. Is this designed to be this way, or are there any good methods to deal with this issue? @MaxHalford

[MaxHalford]

@smastelini care to answer? I think we need a templates answer at some point 😂

[smastelini]

Hi @reasondk, you should take a look at the guide about Hoeffding Trees for more info about memory management.

In short, you can either limit the tree depth or directly pass a memory budget for the model.

[smastelini]

Hey @MaxHalford, maybe we can make some updates on the FAQ page =)

[reasondk]

Thanks. I'll check this guide @MaxHalford @smastelini