README.md

3.3 Observability: Monitoring and alerting

1. Pipeline health monitoring

1. sklearn training pipeline health

---

2. Explainability

Videos

1. Customize layout
2. XGBoost explainability dashboard

Code

• custom/dashboard_data_source.py: this is used to produce the output needed to generate the SHAP values and chart them.
• SHAP values
• SHAP values bar chart
• SHAP values force plot

---

3. Model performance dashboard

Videos

1. Overview dashboard for entire project
2. RMSE and MSE using time series chart
3. RMSE distribution using a histogram
4. Training runs by model using bar chart
5. Training runs by model using pie chart

Code

• Time series RMSE
• Time series MSE
• Histogram
• Runs by model
• data.py

---

4. Alerting

Videos

1. Setup email to send alerts
2. Setup pipeline alerting

Code

• Project metadata.yaml

---

Code

1. sklearn training pipeline health
2. XGBoost explainability dashboard
3. Model performance dashboard configuration

---

Resources

1. How to interpret ML models using SHAP values

2. Customize dashboards for your project and pipelines

3. Confusion matrix

4. Set up alerts for pipeline run statuses:

   1. Email
   2. Opsgenie
   3. Slack
   4. Teams
   5. Discord
   6. Telegram