Skip to content

Commit

Permalink
fix_rendering_first_try
Browse files Browse the repository at this point in the history
  • Loading branch information
@elephaint
elephaint committed Apr 30, 2024
1 parent 47a9716 commit 65133b4
Show file tree
Hide file tree
Showing 2 changed files with 50 additions and 43 deletions.
75 changes: 41 additions & 34 deletions nbs/docs/4_tutorials/loss_function_finetuning.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,16 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"When fine-tuning, the model trains on your dataset to tailor its predictions to your particular scenario. As such, it is possible to specify the loss function used during fine-tuning.\n",
"\n",
"When fine-tuning, the model trains on your dataset to tailor its predictions to your particular scenario. As such, it is possible to specify the loss function used during fine-tuning.\\\n",
"\\\n",
"Specifically, you can choose from:\n",
"- `\"default\"` - a proprietary loss function that is robust to outliers\n",
"- `\"mae\"` - mean absolute error\n",
"- `\"mse\"` - mean squared error\n",
"- `\"rmse\"` - root mean squared error\n",
"- `\"mape\"` - mean absolute percentage error\n",
"- `\"smape\"` - symmetric mean absolute percentage error"
"\n",
"* `\"default\"` - a proprietary loss function that is robust to outliers\n",
"* `\"mae\"` - mean absolute error\n",
"* `\"mse\"` - mean squared error\n",
"* `\"rmse\"` - root mean squared error\n",
"* `\"mape\"` - mean absolute percentage error\n",
"* `\"smape\"` - symmetric mean absolute percentage error"
]
},
{
Expand Down Expand Up @@ -60,7 +61,7 @@
],
"source": [
"#| echo: false\n",
"colab_badge('docs/tutorials/11_loss_function_finetuning')"
"colab_badge('docs/4_tutorials/loss_function_finetuning')"
]
},
{
Expand Down Expand Up @@ -137,8 +138,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's fine-tune the model on a dataset using the mean absolute error (MAE).\n",
"\n",
"Let's fine-tune the model on a dataset using the mean absolute error (MAE).\\\n",
"\\\n",
"For that, we simply pass the appropriate string representing the loss function to the `finetune_loss` parameter of the `forecast` method."
]
},
Expand Down Expand Up @@ -233,9 +234,15 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Fine-tuning with Mean Absolute Error\n",
"Let's fine-tune the model on a dataset using the Mean Absolute Error (MAE).\n",
"\n",
"## 3. Fine-tuning with Mean Absolute Error"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's fine-tune the model on a dataset using the Mean Absolute Error (MAE).\\\n",
"\\\n",
"For that, we simply pass the appropriate string representing the loss function to the `finetune_loss` parameter of the `forecast` method."
]
},
Expand Down Expand Up @@ -294,54 +301,54 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, depending on your data, you will use a specific error metric to accurately evaluate your forecasting model's performance.\n",
"\n",
"Below is a non-exhaustive guide on which metric to use depending on your use case.\n",
"\n",
"**Mean absolute error (MAE)**\n",
"\n",
"Now, depending on your data, you will use a specific error metric to accurately evaluate your forecasting model's performance.\\\n",
"\\\n",
"Below is a non-exhaustive guide on which metric to use depending on your use case.\\\n",
"\\\n",
"**Mean absolute error (MAE)**\\\n",
"\\\n",
"<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{MAE}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} |y_{\\tau} - \\hat{y}_{\\tau}|\">\n",
"\n",
"- Robust to outliers\n",
"- Easy to understand\n",
"- You care equally about all error sizes\n",
"- Same units as your data\n",
"\n",
"**Mean squared error (MSE)**\n",
"\n",
"**Mean squared error (MSE)**\\\n",
"\\\n",
"<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{MSE}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} (y_{\\tau} - \\hat{y}_{\\tau})^{2}\">\n",
"\n",
"- You want to penalize large errors more than small ones\n",
"- Sensitive to outliers\n",
"- Used when large errors must be avoided\n",
"- *Not* the same units as your data\n",
"\n",
"**Root mean squared error (RMSE)**\n",
"\n",
"**Root mean squared error (RMSE)**\\\n",
"\\\n",
"<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{RMSE}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\sqrt{\\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} (y_{\\tau} - \\hat{y}_{\\tau})^{2}}\">\n",
"\n",
"- Brings the MSE back to original units of data\n",
"- Penalizes large errors more than small ones\n",
"\n",
"**Mean absolute percentage error (MAPE)**\n",
"\n",
"**Mean absolute percentage error (MAPE)**\\\n",
"\\\n",
"<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{MAPE}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} \\frac{|y_{\\tau}-\\hat{y}_{\\tau}|}{|y_{\\tau}|}\">\n",
"\n",
"- Easy to understand for non-technical stakeholders\n",
"- Expressed as a percentage\n",
"- Heavier penalty on positive errors over negative errors\n",
"- To be avoided if your data has values close to 0 or equal to 0\n",
"\n",
"**Symmmetric mean absolute percentage error (sMAPE)**\n",
"\n",
"**Symmmetric mean absolute percentage error (sMAPE)**\\\n",
"\\\n",
"<img src=\"https://latex.codecogs.com/svg.image?\\mathrm{SMAPE}_{2}(\\mathbf{y}_{\\tau}, \\mathbf{\\hat{y}}_{\\tau}) = \\frac{1}{H} \\sum^{t+H}_{\\tau=t+1} \\frac{|y_{\\tau}-\\hat{y}_{\\tau}|}{|y_{\\tau}|+|\\hat{y}_{\\tau}|}\">\n",
"\n",
"- Fixes bias of MAPE\n",
"- Equally senstitive to over and under forecasting\n",
"- To be avoided if your data has values close to 0 or equal to 0\n",
"\n",
"With TimeGPT, you can choose your loss function during fine-tuning as to maximize the model's performance metric for your particular use case. \n",
"\n",
"With TimeGPT, you can choose your loss function during fine-tuning as to maximize the model's performance metric for your particular use case.\\\n",
"\\\n",
"Let's run a small experiment to see how each loss function improves their associated metric when compared to the default setting."
]
},
Expand Down Expand Up @@ -650,10 +657,10 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"From the table above, we can see that using a specific loss function during fine-tuning will improve its associated error metric when compared to the default loss function.\n",
"\n",
"In this example, using the MAE as the loss function improves the metric by 8.54% when compared to using the default loss function.\n",
"\n",
"From the table above, we can see that using a specific loss function during fine-tuning will improve its associated error metric when compared to the default loss function.\\\n",
"\\\n",
"In this example, using the MAE as the loss function improves the metric by 8.54% when compared to using the default loss function.\\\n",
"\\\n",
"That way, depending on your use case and performance metric, you can use the appropriate loss function to maximize the accuracy of the forecasts."
]
}
Expand Down
18 changes: 9 additions & 9 deletions nbs/docs/4_tutorials/uncertainty_quantification_with_quantile_forecasts.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@
"id": "0f73b521-05f2-4c53-b44c-d988bf8fe7b9",
"metadata": {},
"source": [
"In forecasting, we are often interested in a distribution of predictions rather than only a point prediction, because we want to have a notion of the uncertainty around the forecast.\n",
"\n",
"To this end, we can create _quantile forecasts_.\n",
"\n",
"Quantile forecasts have an intuitive interpretation, as they present a specific percentile of the forecast distribution. This allows us to make statements such as 'we expect 90% of our observations of air passengers to be above 100'. This approach is helpful for planning under uncertainty, providing a spectrum of possible future values and helping users make more informed decisions by considering the full range of potential outcomes. \n",
"\n",
"With TimeGPT, we can create a distribution of forecasts, and extract the quantile forecasts for a specified percentile. For instance, the 25th and 75th quantiles give insights into the lower and upper quartiles of expected outcomes, respectively, while the 50th quantile, or median, offers a central estimate.\n",
"\n",
"In forecasting, we are often interested in a distribution of predictions rather than only a point prediction, because we want to have a notion of the uncertainty around the forecast.\\\n",
"\\\n",
"To this end, we can create _quantile forecasts_.\\\n",
"\\\n",
"Quantile forecasts have an intuitive interpretation, as they present a specific percentile of the forecast distribution. This allows us to make statements such as 'we expect 90% of our observations of air passengers to be above 100'. This approach is helpful for planning under uncertainty, providing a spectrum of possible future values and helping users make more informed decisions by considering the full range of potential outcomes.\\\n",
"\\\n",
"With TimeGPT, we can create a distribution of forecasts, and extract the quantile forecasts for a specified percentile. For instance, the 25th and 75th quantiles give insights into the lower and upper quartiles of expected outcomes, respectively, while the 50th quantile, or median, offers a central estimate.\\\n",
"\\\n",
"TimeGPT uses [conformal prediction](https://en.wikipedia.org/wiki/Conformal_prediction) to produce the quantiles."
]
},
Expand Down Expand Up @@ -65,7 +65,7 @@
],
"source": [
"#| echo: false\n",
"colab_badge('docs/tutorials/10_uncertainty_quantification_with_quantile_forecasts')"
"colab_badge('docs/4_tutorials/uncertainty_quantification_with_quantile_forecasts')"
]
},
{
Expand Down

0 comments on commit 65133b4

Please sign in to comment.