Merge pull request #768 from rspamd/vstakhov-3.9

Prepare 3.9 documentation

_posts/2024-07-12-rspamd-3.9.0.md

@@ -0,0 +1,43 @@
+---
+layout: post
+title: "Rspamd 3.9.0 has been released"
+categories: announce
+---
+
+Today we have released Rspamd 3.9.0, featuring many new features and fixes. The most important ones are highlighted below. Refer to the [migration notes]({{ site.baseurl }}/doc/tutorials/migration.html) for an overview of potentially-breaking changes.
+
+## Improvements to Bayes configuration
+
+Rspamd now uses a reduced window size of 2 words by default. This change does not require retraining of statistics. In our tests, this reduced window size has produced the equal or better results with better performance and lower storage requirements - around 4 times less than with the previous default window size of 5 words. The new `rspamadm classifier_test` utility can be used for your own experiments.
+
+## New GPT module
+
+This release provides a module for using LLMs for text classification and unsupervised learning. You can read more about it in a [dedicated blog post]({{ site.baseurl }}/misc/2024/07/03/gpt.html).
+
+## Improvements to `known_senders` and `replies` modules
+
+This release introduces enhancements to the `known_senders` and `replies` modules, enabling them to work together to flag verified user contacts. With these improvements, senders to whom a user has previously replied will automatically receive negative scores. For more details, please refer to the documentation of these modules.
+
+## Dynamic multipliers for ratelimits are now disabled by default
+
+To avoid potential confusion, dynamic ratelimits are now disabled by default and must be configured explicitly. Refer to the migration notes for details on how to do this.
+
+## Various bug fixes and new features
+
+* Rspamd HTTP API now supports IO in messagepack serialization format, `rspamc` client uses it now by default
+* Fixed a bug where redis bayes learned cache could grow infinitely (and overflow Redis database)
+* Fixed dynamic_symbols in the multimap plugin
+* Honor dynamic thresholds for greylisting module
+* Fixed a bug with statfiles disabling via settings
+* Fixed slow timer so it is can now distinguish slow sync and async rules and act properly in different cases
+* Implement fuzzy check retransmits backpressure
+* Use libarchive for 7 zip compressed headers
+* Serialize control commands to avoid missing/corrupted transfers over the worker <-> main channels
+* Improved Lua userdata checks performance
+* Reworked `grow_factor` to work in an orderly fashion
+* Fixed `SUBJ_ALL_CAPS` for unicase scripts by `@ikedas`
+* Fixed relearning of Bayes messages by `@aduernberger`
+* Fixed retrieving word count in the antivirus module by `@PxPert`
+* Improvements for rules by `@twesterhever` and `@ishisora`
+
+We extend our gratitude to everyone who contributed to this release.

doc/tutorials/migration.md

@@ -33,6 +33,21 @@ Discover a reliable step-by-step process for upgrading your Rspamd cluster while
 
 10. Repeat the entire process starting from `step 1` for future updates. This approach ensures a smooth and controlled upgrade process that minimizes potential downtime and issues in your production environment.
 
+## Migration to Rspamd 3.9.0
+
+* The `ratelimit` module now operates in non-dynamic mode by default. This change does not affect any existing buckets, as dynamic rates and dynamic bursts will simply be unused in this mode. To retain the old behaviour, please either set the `dynamic_rate_limit` option to `true` (globally for all ratelimit rules) or configure the `ham_factor_rate`/`spam_factor_rate` and/or `ham_factor_burst`/`spam_factor_burst` multipliers for individual rules as needed.
+
+* Bayes statistics now use a reduced window size (2 words), which has proven to be faster and more space-efficient in our tests. Existing statistics can be used without any modifications or relearning. To restore the old behaviour, one can set the following to `local.d/classifier-bayes.conf`:
+
+~~~
+tokenizer {
+  name = "osb";
+  window = 5;
+}
+~~~
+
+However, it is recommended to use the default settings.
+
 ## Migration to Rspamd 3.7.4
 
 The `exclude_private_ips` setting in RBL module no longer exists in this release (and was broken in previous releases), it can be removed from configuration. This setting is equivalent to `exclude_local`.

index.md

@@ -14,22 +14,22 @@ title: Rspamd spam filtering system
 		<div class="container-fluid w-auto mx-2">
 		<div class="row news-row rounded-3 mx-auto my-4 p-3">
 			<div class="col-12 col-sm-10">
-				<h3 style="margin: 0px;">Study: Rspamd and GPT integration</h3>
-				<span class="date"><small>2024-07-03</small></span>
-				<p style="margin-bottom: 0px; margin-top: 5px">We have studied the efficiency of the new GPT plugin to access LLM (large language models) intelligence.</p>
+				<h3 style="margin: 0px;">News: Rspamd 3.9.0 has been released</h3>
+				<span class="date"><small>2024-07-12</small></span>
+				<p style="margin-bottom: 0px; margin-top: 5px">New stable release is now available.</p>
 			</div>
 			<div class="col-12 col-sm-2 d-flex justify-content-end align-items-end">
-				<a class="btn btn-primary mb-2" href="{{ site.baseurl }}/misc/2024/07/03/gpt.html">Learn more&nbsp;<i class="fa-solid fa-chevron-right"></i></a>
+				<a class="btn btn-primary mb-2" href="{{ site.baseurl }}/announce/2024/07/12/rspamd-3.9.0.html">Learn more&nbsp;<i class="fa-solid fa-chevron-right"></i></a>
 			</div>
 		</div>
 		<div class="row news-row rounded-3 mx-auto my-4 p-3">
 			<div class="col-12 col-sm-10">
-				<h3 style="margin: 0px;">News: Rspamd 3.8.4 has been released</h3>
-				<span class="date"><small>2024-02-22</small></span>
-				<p style="margin-bottom: 0px; margin-top: 5px">New stable release is now available.</p>
+				<h3 style="margin: 0px;">Study: Rspamd and GPT integration</h3>
+				<span class="date"><small>2024-07-03</small></span>
+				<p style="margin-bottom: 0px; margin-top: 5px">We have studied the efficiency of the new GPT plugin to access LLM (large language models) intelligence.</p>
 			</div>
 			<div class="col-12 col-sm-2 d-flex justify-content-end align-items-end">
-				<a class="btn btn-primary mb-2" href="{{ site.baseurl }}/announce/2024/02/26/rspamd-3.8.4.html">Learn more&nbsp;<i class="fa-solid fa-chevron-right"></i></a>
+				<a class="btn btn-primary mb-2" href="{{ site.baseurl }}/misc/2024/07/03/gpt.html">Learn more&nbsp;<i class="fa-solid fa-chevron-right"></i></a>
 			</div>
 		</div>
 	</div>