Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-6723 - AdaBoost #15639

Merged
merged 34 commits into from
Sep 26, 2023
Merged

GH-6723 - AdaBoost #15639

merged 34 commits into from
Sep 26, 2023

Conversation

valenad1
Copy link
Collaborator

@valenad1 valenad1 commented Jul 17, 2023 •
edited
Loading

#6723

Implementation of AdaBoost algorithm.

Algorithm input

adaboost_model = H2OAdaBoostEstimator(nlearners = 50, learn_rate = 0.5, weak_learner = "DRF", seed = 0xBEEF)
adaboost_model.train(training_frame=training_frame, y = response)

Whole algorithm output from training

Model Details
=============
H2OAdaBoostEstimator : AdaBoost
Model Key: AdaBoost_model_python_1695210886653_28170


Model Summary: 
    number_of_weak_learners    learn_rate    weak_learner    seed
--  -------------------------  ------------  --------------  ------
    50                         0.5           DRF             48879

Algorithm output from scoring

  predict        p0         p1
        0  0.932313  0.0676868
        1  0.372386  0.627614
        0  0.904304  0.0956963
        0  0.677733  0.322267
        0  0.970847  0.029153
        1  0.372386  0.627614
        1  0.182789  0.817211
        1  0.30957   0.69043
        0  0.918434  0.0815656
        0  0.931758  0.0682418

Computing time benchmark compared to H2O GBM on higgs dataset - training

image

Prediction performance on prostate dataset

Reported on validation dataset

image

@valenad1 valenad1 force-pushed the valenad-6723-adaboost branch 4 times, most recently from a2a2118 to d4c7d53 Compare August 30, 2023 16:13
@valenad1 valenad1 force-pushed the valenad-6723-adaboost branch 5 times, most recently from 6f024ec to df188df Compare September 13, 2023 11:55
@valenad1 valenad1 force-pushed the valenad-6723-adaboost branch 5 times, most recently from 18e679e to 719a356 Compare September 19, 2023 15:58
@maurever
Copy link
Contributor

Good Job @valenad1! I have just a few suggestions.

@wendycwong
Copy link
Contributor

Thank you Adam for the great implementation. I don't have any more suggestions. @maurever : Please approve when you are satisfied. Thanks, Wendy

@maurever maurever requested a review from mn-mikke September 25, 2023 15:41
maurever
maurever previously approved these changes Sep 25, 2023
View reviewed changes
Copy link
Contributor

@maurever maurever left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @valenad1, LGTM.

mn-mikke
mn-mikke previously approved these changes Sep 26, 2023
View reviewed changes
Copy link
Collaborator

@mn-mikke mn-mikke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great change!

@valenad1
add simple model summary
98191ba
@valenad1
fix java api
589033d
@valenad1
Implement possibility to have a custom weights column and ensure that…
b413535 
… the created weights collumn will always be the one we use
@valenad1
Add categorical test
6077424
@valenad1
Cleanup that didn't change tests results
7a65eb7
@valenad1
Remove toCSV since there is API already and commented code
e0d4845
@valenad1
Refactor learning rate to learn rate
f84d9cb
@valenad1
Add documentation and validation to parameters
e3a0309
@valenad1
Add documentation to AdaBoost class
5345b9b
@valenad1
add log
1840cab
@valenad1
Refactor AdaBoost - simple refactor
0485715
@valenad1
Fix GLM as a weak learner
4810d4f
@valenad1
Add GBM as a weak learner
40bca78
@valenad1
test cleanup
32d96b2
@valenad1
Refactor n_estimators to nlearners
8574bd2
@valenad1
fixup! Implement possibility to have a custom weights column and ensu…
bf7a435 
…re that the created weights collumn will always be the one we use
@valenad1
Fix for different model as a weak learner - use upperclass instead of…
445dc39 
… DRF
@valenad1
Ensure that adaboost create exactly nlearners models
4ec9229
@valenad1
Refactoring according to suggestions
e52c287
@valenad1
fixup! Fix for different model as a weak learner - use upperclass ins…
aeaa11b 
…tead of DRF
@valenad1 valenad1 force-pushed the valenad-6723-adaboost branch from 44867f5 to aeaa11b Compare September 26, 2023 14:57
@valenad1
GH-6723 AdaBoost API (#15732)
5b2b780 
* GH-6723 - implement AdaBoost API for python and R

* Expose weights column

* Refactor learning_rate to learn_rate

* Add GBM to the weak_learners

* Refactor n_estimators to nlearners
@valenad1 valenad1 dismissed stale reviews from maurever and mn-mikke via 5b2b780 September 26, 2023 15:02
@valenad1 valenad1 requested a review from wendycwong September 26, 2023 16:34
@valenad1 valenad1 merged commit 010554a into master Sep 26, 2023
2 checks passed
@valenad1 valenad1 deleted the valenad-6723-adaboost branch September 26, 2023 16:45
@valenad1 valenad1 changed the title GH-6723 - Adaboost GH-6723 - AdaBoost Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wendycwong wendycwong wendycwong approved these changes

@mn-mikke mn-mikke mn-mikke approved these changes

@maurever maurever maurever left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@valenad1 @maurever @wendycwong @mn-mikke