pg 35: XGBoost stump severely underperforming scikit-learn decision tree (xgboost version issue)

[nasheqlbrm]

In working through, Chapter 5.4, page 35, I was extremely surprised when I saw the XGBoost Stump underperforming the scikit-learn decision tree with a single node. Essentially the XGBoost Stump always predicts 0 (Data Scientist).

I was finally able to figure out that this is an issue for every XGBoost version > 1.7.6 (I tried up till version 2.1.2). In the screenshots below, the first screenshot is where xgboost version 2.1.2 is used, we can see that the accuracy on the test set is 0.5458. The second screenshot is where xgboost version 1.7.6 is used, we can see that the accuracy score on the test set is 0.6243 as in the book.

I am not sure why this is happening but hope this note helps folks working through the book save some head scratching.

xgboost version 2.1.2

xgboost version 1.7.6