Merge branch 'release-v60.5.0' into release

.buildconfig.yml

@@ -1,4 +1,4 @@
-libraryVersion: 60.4.0
+libraryVersion: 60.5.0
 groupId: org.mozilla.telemetry
 projects:
   glean:

.circleci/config.yml

@@ -90,6 +90,10 @@ commands:
           name: Run Rust RLB delayed ping data test
           command: |
             glean-core/rlb/tests/test-delayed-ping-data.sh
+      - run:
+          name: Run Rust RLB flush test
+          command: |
+            glean-core/rlb/tests/test-ping-lifetime-flush.sh
       - run:
           name: Upload coverage report
           command: |

CHANGELOG.md

@@ -1,6 +1,19 @@
 # Unreleased changes
 
-[Full changelog](https://github.com/mozilla/glean/compare/v60.4.0...main)
+[Full changelog](https://github.com/mozilla/glean/compare/v60.5.0...main)
+
+# v60.5.0 (2024-08-06)
+
+[Full changelog](https://github.com/mozilla/glean/compare/v60.4.0...v60.5.0)
+
+* General
+  * Make auto-flush behavior configurable and time-based ([#2871](https://github.com/mozilla/glean/pull/2871))
+  * Require `glean_parser` v14.5.0 ([#2916](https://github.com/mozilla/glean/pull/2916))
+* Android
+  * Update to Gradle v8.9 ([#2909](https://github.com/mozilla/glean/pull/2909))
+  * Fixed `GleanTestLocalServer` test rule to prevent leaking between tests([Bug 1787234](https://bugzilla.mozilla.org/show_bug.cgi?id=1787234))
+* Rust
+  * Remove cargo feature `preinit_million_queue` and set the default pre-init queue size to 10^6 for all consumers ([Bug 1909246](https://bugzilla.mozilla.org/show_bug.cgi?id=1909246))
 
 # v60.4.0 (2024-07-23)
 

DEPENDENCIES.md

@@ -4951,9 +4951,9 @@ SOFTWARE.
 
 The following text applies to code linked from these dependencies:
 
-* [glean-core 60.4.0]( https://github.com/mozilla/glean )
-* [glean-build 14.3.0]( https://github.com/mozilla/glean )
-* [glean 60.4.0]( https://github.com/mozilla/glean )
+* [glean-core 60.5.0]( https://github.com/mozilla/glean )
+* [glean-build 14.5.1]( https://github.com/mozilla/glean )
+* [glean 60.5.0]( https://github.com/mozilla/glean )
 * [zeitstempel 0.1.1]( https://github.com/badboy/zeitstempel )
 
 ```

Makefile

@@ -151,7 +151,7 @@ docs-python: build-python ## Build the Python documentation
 .PHONY: docs docs-rust docs-swift
 
 docs-metrics: setup-python ## Build the internal metrics documentation
-	$(GLEAN_PYENV)/bin/pip install glean_parser~=14.3
+	$(GLEAN_PYENV)/bin/pip install glean_parser~=14.5
 	$(GLEAN_PYENV)/bin/glean_parser translate --allow-reserved \
 		 -f markdown \
 		 -o ./docs/user/user/collected-metrics \

docs/user/SUMMARY.md

@@ -18,12 +18,6 @@
     - [Validating metrics](user/metrics/validation-checklist.md)
     - [Error reporting](user/metrics/error-reporting.md)
     - [Metrics collected by the Glean SDKs](user/collected-metrics/metrics.md)
-    - [Data Control Plane (Remote Metric Configuration)](user/metrics/data-control-plane/index.md)
-      - [Example Scenarios](user/metrics/data-control-plane/example-scenarios.md)
-      - [Product Integration](user/metrics/data-control-plane/product-integration.md)
-      - [Experimenter Configuration](user/metrics/data-control-plane/experimenter-configuration.md)
-      - [Advanced Topics](user/metrics/data-control-plane/advanced-topics.md)
-      - [Frequently Asked Questions](user/metrics/data-control-plane/faq.md)
 - [Pings](user/pings/index.md)
     - [Adding new custom pings](user/pings/custom.md)
     - [Testing custom pings](user/pings/testing-custom-pings.md)
@@ -33,9 +27,18 @@
         - [Events Ping](user/pings/events.md)
         - [Metrics Ping](user/pings/metrics.md)
         - [Schedules and timings overview](user/pings/ping-schedules-and-timings.md)
-    - [Data Control Plane (Remote Metric Configuration)](user/pings/data-control-plane/index.md)
-      - [Product Integration](user/pings/data-control-plane/product-integration.md)
-      - [Experimenter Configuration](user/pings/data-control-plane/experimenter-configuration.md)
+- [Server Knobs: Glean Data Control Plane](user/server-knobs/index.md)
+  - [Controlling Metrics with Server Knobs](user/server-knobs/metrics/index.md)
+    - [Example Scenarios](user/server-knobs/metrics/example-scenarios.md)
+    - [Product Integration](user/server-knobs/metrics/product-integration.md)
+    - [Experimenter Configuration](user/server-knobs/metrics/experimenter-configuration.md)
+    - [Advanced Topics](user/server-knobs/metrics/advanced-topics.md)
+    - [Frequently Asked Questions](user/server-knobs/metrics/faq.md)
+  - [Controlling Pings with Server Knobs](user/server-knobs/pings/index.md)
+    - [Product Integration](user/server-knobs/pings/product-integration.md)
+    - [Experimenter Configuration](user/server-knobs/pings/experimenter-configuration.md)
+  - [Other Server Knobs](user/server-knobs/other/index.md)
+    - [Max Events per Ping](user/server-knobs/other/max-events.md)
 - [Debugging products using Glean](user/debugging/index.md)
     - [Android](user/debugging/android.md)
     - [iOS](user/debugging/ios.md)

docs/user/reference/general/initializing.md

@@ -16,7 +16,7 @@ application start using Glean.
 ## Behavior when uninitialized
 
 Any API called before Glean is initialized is queued and applied at initialization.
-To avoid unbounded memory growth the queue is bounded (currently to a maximum of 100 tasks),
+To avoid unbounded memory growth the queue is bounded (currently to a maximum of 1 million tasks),
 and further calls are dropped.
 
 The number of calls dropped, if any,

docs/user/reference/metrics/event.md

@@ -394,7 +394,8 @@ Each extra key contains additional metadata:
 * In Glean.js the default value for `maxEvents` is 1. In all other SDKs it is 500.
 * Once the `maxEvents` threshold is reached on the client an "events" ping is immediately sent.
 * The `extra_keys` allows for a maximum of 50 keys.
-* The keys in the `extra_keys` list must be in dotted snake case, with a maximum length of 40 bytes, when encoded as UTF-8.
+* The keys in the `extra_keys` list must be written using printable ASCII characters,
+  with a maximum length of 40 bytes, when encoded as UTF-8.
 * The values in the `extras` object have a maximum length of 500 bytes when serialized and encoded as UTF-8.
   Longer values are truncated, and an `invalid_overflow` error is recorded.
 

docs/user/user/server-knobs/index.md

@@ -0,0 +1,25 @@
+# Server Knobs: Glean Data Control Plane
+
+Glean provides Server Knobs, a Data [Control Plane] through which Glean runtime settings can be changed remotely including the ability to enable, disable or throttle metrics and pings through a [Nimbus] rollout or experiment.
+
+Products can use this capability to control "data traffic", similar to how a network control plane controls "network traffic".
+
+Server Knobs provides the ability to do the following:
+
+- Allow runtime changes to data collection without needing to land code and ride release trains.
+- Eliminate the need for manual creation and maintenance of feature flags specific to data collection.
+- Sampling of measurements from a subset of the population so that we do not collect or ingest more data than is necessary from high traffic areas of an application instrumented with Glean metrics.
+- Operational safety through being able to react to high-volume or unwanted data.
+- Visibility into sampling and sampling rates for remotely configured metrics.
+
+## Contents
+
+- [Controlling Metrics with Server Knobs]
+- [Controlling Pings with Server Knobs]
+- [Other Server Knobs]
+
+[Control Plane]: https://en.wikipedia.org/wiki/Control_plane
+[Nimbus]: https://experimenter.info
+[Controlling Metrics with Server Knobs]: ./metrics/index.md
+[Controlling Pings with Server Knobs]: ./pings/index.md
+[Other Server Knobs]: ./other/index.md

docs/user/user/metrics/data-control-plane/index.md → docs/user/user/server-knobs/metrics/index.md

@@ -23,4 +23,4 @@ For information on controlling pings with Server Knobs, see the metrics document
 
 [Control Plane]: https://en.wikipedia.org/wiki/Control_plane
 [Nimbus]: https://experimenter.info
-[Server Knobs - Pings]: ../../pings/data-control-plane/index.md
+[Server Knobs - Pings]: ../pings/index.md

docs/user/user/metrics/data-control-plane/product-integration.md → docs/user/user/server-knobs/metrics/product-integration.md

@@ -8,14 +8,18 @@ In order to make use of the remote metric configuration in a Firefox Desktop com
 
 ### Integration Option 1:
 
-Glean provides a general Nimbus feature that can be used for configuration of metrics named `glean`. Simply select the `glean` feature along with your Nimbus feature from the Experimenter UI when configuring the experiment or rollout (see https://experimenter.info for more information on multi-feature experiments).
+Glean provides a general Nimbus feature that can be used for configuration of metrics named `glean`. Simply select the `glean` feature along with your Nimbus feature from the Experimenter UI when configuring the experiment or rollout (see [Nimbus] documentation for more information on multi-feature experiments).
 
-The `glean` Nimbus feature requires the `gleanMetricConfiguration` variable to be used to provide the require metric configuration. The format of the configuration is a map of the fully qualified metric name (category.name) of the metric to a boolean value representing whether the metric is enabled. If a metric is omitted from this map, it will default to the value found in the metrics.yaml. An example configuration for the `glean` feature can be found on the [Experimenter Configuration](./experimenter-configuration.md) page.
+The `glean` Nimbus feature requires the `gleanMetricConfiguration` variable to be used to provide the required metric configuration.
+The format of the configuration is a map of the fully qualified metric name (category.name) of the metric to a boolean value representing whether the metric is enabled.
+If a metric is omitted from this map, it will default to the value found in the metrics.yaml.
+An example configuration for the `glean` feature can be found on the [Experimenter Configuration] page.  
 
 
 ### Integration Option 2 (Advanced use):
 
- A second option that can give you more control over the metric configuration, especially if there are more than one experiments or rollouts that are currently using the `glean` feature from Option 1 above, is to add a Feature Variable to represent the Glean metric configuration in your own feature. This can be accomplished by modifying the FeatureManifest.yaml file, adding a variable through which to pass metric configurations. Glean will handle merging this configuration with other metrics configurations for you (See [Advanced Topics](./advanced-topics.md) for more info on this). An example feature manifest entry would look like the following:
+ A second option that can give you more control over the metric configuration, especially if there are more than one experiments or rollouts that are currently using the `glean` feature from Option 1 above, is to add a Feature Variable to represent the Glean metric configuration in your own feature. This can be accomplished by modifying the FeatureManifest.yaml file, adding a variable through which to pass metric configurations. Glean will handle merging this configuration with other metrics configurations for you (See [Advanced Topics] for more info on this).
+ An example feature manifest entry would look like the following:
 
 ```yaml
 variables:
@@ -26,7 +30,8 @@ variables:
     "Glean metric configuration"
 ```
 
-This definition allows for configuration to be set in a Nimbus rollout or experiment and fetched by the client to be applied based on the enrollment. Once the Feature Variable has been defined, the final step is to fetch the configuration from Nimbus and supply it to the Glean API. This can be done during initialization and again any time afterwards, such as in response to receiving an updated configuration from Nimbus. Glean will merge this configuration with any other active configurations and enable or disable the metrics accordingly. An example call to set a configuration through your Nimbus Feature could look like this:
+This definition allows for configuration to be set in a Nimbus rollout or experiment and fetched by the client to be applied based on the enrollment. Once the Feature Variable has been defined, the final step is to fetch the configuration from Nimbus and supply it to the Glean API. This can be done during initialization and again any time afterwards, such as in response to receiving an updated configuration from Nimbus. Glean will merge this configuration with any other active configurations and enable or disable the metrics accordingly.
+An example call to set a configuration through your Nimbus Feature could look like this:
 
 ```JavaScript
 // Fetch the Glean metric configuration from your feature's Nimbus variable
@@ -55,13 +60,18 @@ lazy.NimbusFeatures.yourNimbusFeatureName.onUpdate(() => {
 
 ### Integration Option 1:
 
-Glean provides a general Nimbus feature that can be used for configuration of metrics named `glean`. Simply select the `glean` feature along with your Nimbus feature from the Experimenter UI when configuring the experiment or rollout (see https://experimenter.info for more information on multi-feature experiments).
+Glean provides a general Nimbus feature that can be used for configuration of metrics named `glean`. Simply select the `glean` feature along with your Nimbus feature from the Experimenter UI when configuring the experiment or rollout (see [Nimbus] documentation for more information on multi-feature experiments).
 
-The `glean` Nimbus feature requires the `gleanMetricConfiguration` variable to be used to provide the require metric configuration. The format of the configuration is a map of the fully qualified metric name (category.name) of the metric to a boolean value representing whether the metric is enabled. If a metric is omitted from this map, it will default to the value found in the metrics.yaml. An example configuration for the `glean` feature can be found on the [Experimenter Configuration](./experimenter-configuration.md) page.
+The `glean` Nimbus feature requires the `gleanMetricConfiguration` variable to be used to provide the required metric configuration.
+The format of the configuration is a map of the fully qualified metric name (category.name) of the metric to a boolean value representing whether the metric is enabled.
+If a metric is omitted from this map, it will default to the value found in the metrics.yaml.
+An example configuration for the `glean` feature can be found on the [Experimenter Configuration] page.  
 
 ### Integration Option 2 (Advanced use):
 
-A second option that can give you more control over the metric configuration, especially if there are more than one experiments or rollouts that are currently using the `glean` feature from Option 1 above, is to add a Feature Variable to represent the Glean metric configuration in your own feature. This can be accomplished by modifying the Nimbus Feature Manifest file, adding a variable through which to pass metric configurations. Glean will handle merging this configuration with other metrics configurations for you (See [Advanced Topics](./advanced-topics.md) for more info on this). An example feature manifest entry would look like the following:
+A second option that can give you more control over the metric configuration, especially if there are more than one experiments or rollouts that are currently using the `glean` feature from Option 1 above, is to add a Feature Variable to represent the Glean metric configuration in your own feature.
+This can be accomplished by modifying the Nimbus Feature Manifest file, adding a variable through which to pass metric configurations. Glean will handle merging this configuration with other metrics configurations for you (See [Advanced Topics] for more info on this).
+An example feature manifest entry would look like the following:
 
 ```yaml
 features:
@@ -77,7 +87,8 @@ features:
         default: "{}"
 ```
 
-Once the Feature Variable has been defined, the final step is to fetch the configuration from Nimbus and supply it to the Glean API. This can be done during initialization and again any time afterwards, such as in response to receiving an updated configuration from Nimbus. Only the latest configuration provided will be applied and any previously configured metrics that are omitted from the new configuration will not be changed. An example call to set a configuration from the “homescreen” Nimbus Feature could look like this:
+Once the Feature Variable has been defined, the final step is to fetch the configuration from Nimbus and supply it to the Glean API. This can be done during initialization and again any time afterwards, such as in response to receiving an updated configuration from Nimbus. Only the latest configuration provided will be applied and any previously configured metrics that are omitted from the new configuration will not be changed.
+An example call to set a configuration from the “homescreen” Nimbus Feature could look like this:
 
 ```Swift
 Glean.applyServerKnobsConfig(FxNimbus.features.homescreen.value().metricsEnabled)
@@ -86,4 +97,6 @@ Glean.applyServerKnobsConfig(FxNimbus.features.homescreen.value().metricsEnabled
 Since mobile experiments only update on initialization of the application, it isn't necessary to register to listen for notifications for experiment updates.
 
 [Nimbus]: https://experimenter.info
-[Nimbus Desktop Feature API]: https://experimenter.info/desktop-feature-api
+[Nimbus Desktop Feature API]: https://experimenter.info/desktop-feature-api
+[Advanced Topics]: ./advanced-topics.md
+[Experimenter Configuration]: ./experimenter-configuration.md

docs/user/user/server-knobs/other/index.md

@@ -0,0 +1,15 @@
+# Other Server Knobs
+
+Below are additional Glean parameters and settings that are exposed via Server Knobs for use in a [Nimbus] experiment or rollout.
+
+## Contents
+- [Max Events per Event Ping]
+
+Additional Glean settings will be added to Server Knobs as needed or by request.
+
+> *For information on controlling metrics and pings via Server Knobs, please refer to [Controlling Metrics with Server Knobs] and [Controlling Pings with Server Knobs].*
+
+[Nimbus]: https://experimenter.info
+[Controlling Metrics with Server Knobs]: ../metrics/index.md
+[Controlling Pings with Server Knobs]: ../pings/index.md
+[Max Events per Event Ping]: ./max-events.md

docs/user/user/server-knobs/other/max-events.md

@@ -0,0 +1,16 @@
+# Max Events
+
+By default, Glean batches events together to submit on a single events ping.
+The `event_threshold` Server Knob controls how many events Glean will collect before submitting an events ping.
+
+For instance, if you wanted to disable batching in order to transmit an events ping after every event is recorded you could set `event_threshold: 1`.
+
+## Example Configuration:
+
+```json
+{
+  "gleanMetricConfiguration": {
+    "event_threshold": 1
+  }
+}
+```