Skip to content

Submission Scores

Jump to bottom
Adrodoc edited this page Dec 13, 2019 · 11 revisions

This page serves for sharing and documenting different training approaches.

  • Score: 0.99 - LEAKS
  • Model: LightGBM 2-Fold
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse"
}
  • Training: 500 rds, no early stopping
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure, month;
  • Score: 1.06
  • Model: LightGBM 4-Fold
  • Parameters:
lgbm_params:
  objective: regression
  boosting: dart
  learning_rate: 0.05
  bagging_fraction: 0.9484361954387427
  feature_fraction: 0.8748759225130817
  min_data_in_leaf: 30
  min_split_gain: 0.2335888161661162
  num_leaves: 3480
  reg_alpha: 0.2748100019294575
  reg_lambda: 3.8233940707366925
  metric: rmse
  verbosity: -1

num_boost_round: 750
verbose_eval: 50
splits_for_cv: 4
  • Training: inf rds, 50 rds early stopping
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure, month; results of hyperopt
  • Score: 1.07
  • Model: LightGBM 4-Fold
  • Parameters:
lgbm_params:
  objective: regression
  boosting: gbdt
  learning_rate: 0.05
  bagging_fraction: 0.7604703318859309
  feature_fraction: 0.9556886502673504
  min_data_in_leaf: 15
  min_split_gain: 0.4673249117301946
  num_leaves: 3480
  reg_alpha: 3.9145881110580314
  reg_lambda: 1.2467155815765916
  metric: rmse
  verbosity: -1

num_boost_round: 99999
early_stopping_rounds: 50
verbose_eval: 50
splits_for_cv: 4
  • Training: inf rds, 50 rds early stopping
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure, month; results of hyperopt
  • Score: 1.08
  • Model: CatBoost 4-Fold
  • Parameters:
ctb_params:
  iterations: 99999
  learning_rate: 0.05
  reg_lambda: 2
  loss_function: RMSE
  task_type: GPU

ctb_early_stopping_rounds: 20
ctb_verbose_eval: 25
ctb_splits_for_cv: 4
  • Training: inf rds, 20 rds early stopping
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure, month;
  • Score: 1.08
  • Model: LightGBM 3-Fold
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 1250,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse"
}
  • Training: 1000 rds, 100rds early stopping
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure, month;
  • Score: 1.08
  • Average Eval Loss: 0.847
  • Average Train Loss: 0.693
  • Commit: 45a3e80
  • Model: LightGBM
  • Mode: cv (3 folds)
  • Training: 1000 rds
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse",
    "verbosity": -1
}

Output: https://github.com/energeeks/ashrae-energy-prediction/issues/57#issuecomment-560019127

  • Score: 1.10
  • Average Eval Loss: 0.575
  • Average Train Loss: 0.418
  • Commit: 826755b
  • Model: LGBM
  • Mode: daywise_cv
  • With hourofyear column
  • Parameters:
lgbm_params:
  objective: regression
  boosting: gbdt
  learning_rate: 0.05
  bagging_fraction: 0.7604703318859309
  feature_fraction: 0.9556886502673504
  min_data_in_leaf: 15
  min_split_gain: 0.4673249117301946
  num_leaves: 3480
  reg_alpha: 3.9145881110580314
  reg_lambda: 1.2467155815765916
  metric: rmse
  verbosity: -1
  
lgbm_num_boost_round: 999999
lgbm_early_stopping_rounds: 50
lgbm_verbose_eval: 25
lgbm_splits_for_cv: 2
  • Score: 1.10
  • Model: LightGBM 2-Fold
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse"
}
  • Training: 500 rds, no early stopping
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure, month;
  • Score: 1.11
  • Model: LightGBM 2-Fold
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse"
}
  • Training: 2000/500 rds, no early stopping (2k rds better position in lb)
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure;
  • Score: 1.12
  • Model: LightGBM 3-Fold
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse"
}
  • Training: 1000 rds / 100 early stopping
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure;
  • Score: 1.30
  • Average Eval Loss: 0.890
  • Average Train Loss: 0.831
  • Commit: e2d9134 (split using weekofyear instead of dayofyear)
  • Model: LightGBM
  • Mode: daywise_cv
  • Training: 500 rds, no early stopping
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse",
    "verbosity": -1
}

Output: https://github.com/energeeks/ashrae-energy-prediction/issues/44#issuecomment-557613284

  • Score: 1.31
  • Average Eval Loss: 0.878
  • Average Train Loss: 0.845
  • Commit: 3bb1a7a
  • Model: LightGBM
  • Mode: daywise_cv
  • Training: 500 rds, no early stopping
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse",
    "verbosity": -1
}

Output: https://github.com/energeeks/ashrae-energy-prediction/issues/44#issuecomment-557535405

  • Score: 1.36
  • Model: LightGBM BY METER
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse"
}
  • Training: 500 rds, no early stopping
  • Features: NO OHE/Circular; Dropped: Wind cols, seapressure;
  • Score: 2.61
  • Model: LightGBM 2-Fold - Training per Building
  • Parameters: 
    params = {
    "objective": "regression",
    "boosting": "gbdt",
    "num_leaves": 40,
    "learning_rate": 0.05,
    "feature_fraction": 0.85,
    "reg_lambda": 2,
    "metric": "rmse"
}
  • Training: 500 rds, no early stopping
  • Features: NO OHE/Circular; Dropped Wind cols, seapressure, month, site_id, building_id;
Clone this wiki locally