Merge branch 'release-v60.4.0' into release

.buildconfig.yml

@@ -1,4 +1,4 @@
-libraryVersion: 60.3.0
+libraryVersion: 60.4.0
 groupId: org.mozilla.telemetry
 projects:
   glean:

.circleci/config.yml

@@ -62,18 +62,18 @@ commands:
       - skip-if-doc-only
       - setup-rust-toolchain:
           rust-version: <<parameters.rust-version>>
-      - run:
-          name: Test
-          command: |
-            export GLEAN_TEST_COVERAGE=$(realpath glean_coverage.txt)
-            cargo test --verbose --jobs 6 -- --nocapture
       - run:
           name: Install required Python dependencies
           command: |
             sudo apt update
             sudo apt install --yes --no-install-recommends \
               python3-pip \
               python3.10-venv
+      - run:
+          name: Test
+          command: |
+            export GLEAN_TEST_COVERAGE=$(realpath glean_coverage.txt)
+            cargo test --workspace --verbose --jobs 6 -- --nocapture
       - run:
           name: Run Rust sample
           command: |
@@ -86,6 +86,10 @@ commands:
           name: Run Rust RLB crash test
           command: |
             glean-core/rlb/tests/test-thread-crashing.sh
+      - run:
+          name: Run Rust RLB delayed ping data test
+          command: |
+            glean-core/rlb/tests/test-delayed-ping-data.sh
       - run:
           name: Upload coverage report
           command: |

.dictionary

@@ -1,4 +1,4 @@
-personal_ws-1.1 en 282 utf-8
+personal_ws-1.1 en 289 utf-8
 AAR
 AARs
 ABI
@@ -108,13 +108,15 @@ XCFramework
 Xcode
 YAML
 aarch
+addon
 alphanumerics
 analytics
 anonymized
 aspell
 async
 autodetected
 backend
+backgrounding
 backoff
 backport
 barcode
@@ -127,6 +129,7 @@ booleans
 brizental
 bugfixes
 camelCase
+cancelled
 carthage
 changelog
 chutten
@@ -147,6 +150,7 @@ datetime
 destructor
 deterministically
 dev
+devtools
 dexter
 distributable
 docstrings
@@ -175,6 +179,7 @@ html
 iMacs
 illumos
 init
+initializer
 inlined
 instrumentations
 integrations
@@ -223,6 +228,7 @@ preinit
 py
 pytest
 rethrow
+retransmission
 rfloor
 rkv
 rkv's
@@ -270,6 +276,7 @@ urlbar
 validator
 vendored
 vendoring
+virtualenv
 walkthrough
 walkthroughs
 webextension

CHANGELOG.md

@@ -1,6 +1,24 @@
 # Unreleased changes
 
-[Full changelog](https://github.com/mozilla/glean/compare/v60.3.0...main)
+[Full changelog](https://github.com/mozilla/glean/compare/v60.4.0...main)
+
+# v60.4.0 (2024-07-23)
+
+[Full changelog](https://github.com/mozilla/glean/compare/v60.3.0...v60.4.0)
+
+* General
+  * Bump the string length limit to 255 characters ([#2857](https://github.com/mozilla/glean/pull/2857))
+  * New metric `glean.database.write_time` to measure database writes ([#2845](https://github.com/mozilla/glean/pull/2845))
+  * Require glean_parser v14.3.0 ([bug 1909244](https://bugzilla.mozilla.org/show_bug.cgi?id=1909244))
+* Android
+  * Delay log init until Glean is getting initialized ([#2858](https://github.com/mozilla/glean/pull/2858))
+  * Update to Gradle v8.8 ([#2860](https://github.com/mozilla/glean/pull/2860))
+  * Updated Kotlin to version 1.9.24 ([#2861](https://github.com/mozilla/glean/pull/2861))
+  * Default-enable `delayPingLifetimeIo` ([#2863](https://github.com/mozilla/glean/issues/2863))
+  * Preparing Glean to be able to remove `service-glean` from Android Components ([#2891](https://github.com/mozilla/glean/pull/2891))
+  * Gradle Plugin: Support for using an external Python environment ([#2889](https://github.com/mozilla/glean/pull/2889))
+* Rust
+  * New Metric Types: `labeled_custom_distribution`, `labeled_memory_distribution`, and `labeled_timing_distribution` ([bug 1657947](https://bugzilla.mozilla.org/show_bug.cgi?id=1657947))
 
 # v60.3.0 (2024-05-31)
 
@@ -35,6 +53,15 @@
 * Python
   * Replace use of deprecated functionality (and make installs work on Python 3.12) ([#2820](https://github.com/mozilla/glean/pull/2820))
 
+# v60.0.1 (2024-05-31)
+
+[Full changelog](https://github.com/mozilla/glean/compare/v60.0.0...v60.0.1)
+
+* Android
+  * Allow configuring `delayPingLifetimeIo` in Kotlin and auto-flush this data after 1000 writes.
+    It is also auto-flushed on background. ([#2851](https://github.com/mozilla/glean/pull/2851))
+    (Backported changes)
+
 # v60.0.0 (2024-04-22)
 
 [Full changelog](https://github.com/mozilla/glean/compare/v59.0.0...v60.0.0)

DEPENDENCIES.md

@@ -4951,9 +4951,9 @@ SOFTWARE.
 
 The following text applies to code linked from these dependencies:
 
-* [glean-core 60.3.0]( https://github.com/mozilla/glean )
-* [glean-build 14.0.1]( https://github.com/mozilla/glean )
-* [glean 60.3.0]( https://github.com/mozilla/glean )
+* [glean-core 60.4.0]( https://github.com/mozilla/glean )
+* [glean-build 14.3.0]( https://github.com/mozilla/glean )
+* [glean 60.4.0]( https://github.com/mozilla/glean )
 * [zeitstempel 0.1.1]( https://github.com/badboy/zeitstempel )
 
 ```

Makefile

@@ -151,7 +151,7 @@ docs-python: build-python ## Build the Python documentation
 .PHONY: docs docs-rust docs-swift
 
 docs-metrics: setup-python ## Build the internal metrics documentation
-	$(GLEAN_PYENV)/bin/pip install glean_parser~=14.0
+	$(GLEAN_PYENV)/bin/pip install glean_parser~=14.3
 	$(GLEAN_PYENV)/bin/glean_parser translate --allow-reserved \
 		 -f markdown \
 		 -o ./docs/user/user/collected-metrics \

docs/dev/SUMMARY.md

@@ -10,6 +10,7 @@
 - [Android bindings](android/index.md)
     - [Setup Build Environment](android/setup-android-build-environment.md)
     - [Android SDK/NDK versions](android/sdk-ndk-versions.md)
+    - [Logging](android/logging.md)
     - [Development with android-components](android/development-with-android-components.md)
     - [Locally-published components in Fenix](android/locally-published-components-in-fenix.md)
     - [Substituting glean_parser](android/glean-parser-substitution.md)

docs/dev/android/logging.md

@@ -0,0 +1,12 @@
+# Logging
+
+Logs from `glean-core` are only passed through to the Android logging framework when `Glean.initialize` is called for the first time.
+This means any logging that might happen before that, e.g. from early metric collection will not be collected.
+
+If these logs are needed for debugging add the following initializer to `Glean.kt`:
+
+```kotlin
+init {
+    gleanEnableLogging()
+}
+```

docs/user/SUMMARY.md

@@ -70,12 +70,15 @@
     - [String List](reference/metrics/string_list.md)
     - [Timespan](reference/metrics/timespan.md)
     - [Timing Distribution](reference/metrics/timing_distribution.md)
+    - [Labeled Timing Distributions](reference/metrics/labeled_timing_distributions.md)
     - [Memory Distribution](reference/metrics/memory_distribution.md)
+    - [Labeled Memory Distributions](reference/metrics/labeled_memory_distributions.md)
     - [UUID](reference/metrics/uuid.md)
     - [URL](reference/metrics/url.md)
     - [Datetime](reference/metrics/datetime.md)
     - [Event](reference/metrics/event.md)
     - [Custom Distribution](reference/metrics/custom_distribution.md)
+    - [Labeled Custom Distributions](reference/metrics/labeled_custom_distributions.md)
     - [Quantity](reference/metrics/quantity.md)
     - [Rate](reference/metrics/rate.md)
     - [Text](reference/metrics/text.md)

docs/user/appendix/twig.md

@@ -67,3 +67,4 @@ They could be release notes, documentation, hopes, dreams, or whatever: so long
 * 2022-02-25: [Your personal Glean data pipeline](https://blog.mozilla.org/data/2022/02/25/this-week-in-glean-your-personal-glean-data-pipeline/)
 * 2022-10-27: [Page Load Data, Three Ways (Or, How Expensive Are Events?)](https://blog.mozilla.org/data/2022/10/27/this-week-in-glean-page-load-data-three-ways-or-how-expensive-are-events/)
 * 2023-05-25: [Reading “The Manager’s Path” by Camille Fournier](https://blog.mozilla.org/data/2023/05/25/this-week-in-data-reading-the-managers-path-by-camille-fournier/)
+* 2024-06-27: [Cosmic Rays From Outer Space! (What Comes Next?)](https://blog.mozilla.org/data/2024/06/27/this-week-in-data-cosmic-rays-from-outer-space-what-comes-next/)

docs/user/language-bindings/android/android-build-configuration-options.md

@@ -80,3 +80,13 @@ ext.gleanExpireByVersion = 25
 Different products have different ways to compute the product version at build-time.
 For this reason the Glean Gradle plugin cannot provide an automated way to detect the product major version at build time.
 When using the expiration by version feature in Android, products must provide the major version by themselves.
+
+## `gleanPythonEnvDir`
+
+By default, the Glean Gradle plugin will manage its own Python virtualenv in `$gradleUserHomeDir/glean` to install `glean_parser`. By specifying a path in `ext.gleanPythonEnvDir` you can reuse an existing Python virtualenv.
+
+```groovy
+ext.gleanPythonEnvDir = "$buildDir/externallyManagedVenv"
+```
+
+`glean_parser` must be available in that virtualenv, the Gradle plugin will make no attempt at installing it.

docs/user/language-bindings/android/android-offline-builds.md

@@ -6,7 +6,15 @@ The Glean Kotlin SDK uses a Python script, [`glean_parser`](https://github.com/m
 
 For offline builds, the Python environment, and packages of `glean_parser` and its dependencies must be provided prior to building the Glean-using application.
 
-To build a Glean-using application in offline mode, do the following:
+To build a Glean-using application in offline mode, you can either:
+
+### Provide an externally-managed virtualenv
+
+Set `ext.gleanPythonEnvDir` to your existing virtualenv before applying the plugin, see [`gleanPythonEnvDir`](./android-build-configuration-options.md#gleanpythonenvdir).
+
+### Provide a Python interpreter and the required wheels
+
+In this mode Glean will setup its own virtualenv in `$gradleUserHomeDir/glean`, controlled by the `GLEAN_PYTHON` and `GLEAN_PYTHON_WHEELS_DIR` environment variables.
 
 - Install Python 3.8 or later and ensure it's on the `PATH`.
 

docs/user/reference/general/initializing.md

@@ -56,22 +56,25 @@ May only be called once. Subsequent calls to `initialize` are no-op.
 
 The available `initialize` configuration options may vary depending on the SDK.
 Below are listed the configuration options available on most SDKs.
-
-- `applicationId`: Application identifier. For Android and iOS applications, this is the id used on the platform's respective app store and is extracted automatically from the application context.
-- `uploadEnabled`: The user preference on whether or not data upload is enabled.
-- `appChannel`: The application's release channel. When present, the `app_channel` will be reported in all ping's [`client_info`](../../user/pings/index.html#the-client_info-section) section.
-- `appBuild`: A build identifier e.g. the build identifier generated by a CI system (e.g. "1234/A"). If not present, `app_build` will be reported as "Unknown" on all pings [`client_info`](../../user/pings/index.html#the-client_info-section) section.
-- `appDisplayVersion`: The user visible version string for the application running Glean. If not present, `app_display_version` will be reported as "Unknown" on all pings [`client_info`](../../user/pings/index.html#the-client_info-section) section.
-- `serverEndpoint`: The server pings are sent to. Defaults to `https://incoming.telemetry.mozilla.org`.
-- `maxEvents`: The maximum number of events the Glean storage will hold on to before submitting the 'events' ping. Defaults to 1 for Glean.js, 500 for all other SDKs. Refer to the [`events` ping documentation](../../user/pings/events.md) for more information on its scheduling.
-- `httpUploader`: A custom HTTP uploader instance, that will overwrite Glean's provided uploader. Useful
-for users that wish to use specific uploader implementations. See [Custom Uploaders](#custom-uploaders)
-for more information on how and when the use this feature.
-- `logLevel`: The level for how verbose the internal logging is. The level filter options in order from least to most verbose are: `Off`, `Error`, `Warn`, `Info`, `Debug`, `Trace`. See the [`log` crate docs](https://docs.rs/log/latest/log/) for more information.
-- `rateLimit`: Optional.
-  Specifies the maximum number of pings that can be uploaded per interval of a specified number of seconds.
-  Default is to use the SDK default (presently 15 pings per 60s interval).
-- `experimentationId`: Optional. An identifier derived by the application to be sent in all pings for the purpose of experimentation. See the experiments API documentation for more information.
+Note that on some SDKs some of the options are taken as a configuration object.
+Check the respective SDK documentation for details.
+
+| Configuration Option | Default value | Description |
+| -------------------- | ------------- | ----------- |
+| `applicationId` | On Android/iOS: determined automatically. Otherwise *required*. | Application identifier. For Android and iOS applications, this is the id used on the platform's respective app store and is extracted automatically from the application context. |
+| `uploadEnabled` | _Required_ | The user preference on whether or not data upload is enabled. |
+| `channel` | - | The application's release channel. When present, the `app_channel` will be reported in all pings' [`client_info`](../../user/pings/index.html#the-client_info-section) sections. |
+| `appBuild` | On Android/iOS: determined automatically. Otherwise: - | A build identifier e.g. the build identifier generated by a CI system (e.g. "1234/A"). If not present, `app_build` will be reported as "Unknown" on all pings' [`client_info`](../../user/pings/index.html#the-client_info-section) sections. |
+| `appDisplayVersion` | - | The user visible version string for the application running Glean. If not present, `app_display_version` will be reported as "Unknown" on all pings' [`client_info`](../../user/pings/index.html#the-client_info-section) sections.
+| `serverEndpoint` | `https://incoming.telemetry.mozilla.org` | The server pings are sent to.
+| `maxEvents` | Glean.js: 1. <br> Other SDKs: 500. | The maximum number of events the Glean storage will hold on to before submitting the 'events' ping. Refer to the [`events` ping documentation](../../user/pings/events.md) for more information on its scheduling. |
+| `httpUploader` | - | A custom HTTP uploader instance, that will overwrite Glean's provided uploader. Useful for users that wish to use specific uploader implementations. See [Custom Uploaders](#custom-uploaders) for more information on how and when the use this feature. |
+| `logLevel` | - | The level for how verbose the internal logging is. The level filter options in order from least to most verbose are: `Off`, `Error`, `Warn`, `Info`, `Debug`, `Trace`. See the [`log` crate docs](https://docs.rs/log/latest/log/) for more information. |
+| `enableEventTimestamps` | `true` | Whether to add a wall clock timestamp to all events. |
+| `rateLimit` | 15 pings per 60s interval | Specifies the maximum number of pings that can be uploaded per interval of a specified number of seconds. |
+| `experimentationId` | - | Optional. An identifier derived by the application to be sent in all pings for the purpose of experimentation. See the experiments API documentation for more information. |
+| `enableInternalPings` | `true` | Whether to enable the internal "baseline", "events", and "metrics" pings. |
+| `delayPingLifetimeIo` | `false` | Whether Glean should delay persistence of data from metrics with `ping` lifetime. On Android data is automatically persisted every 1000 writes and on backgrounding when enabled. |
 
 To learn about SDK specific configuration options available, refer to the [Reference](#reference) section.
 
@@ -112,6 +115,7 @@ class SampleApplication : Application() {
             // Here, `settings()` is a method to get user preferences, specific to
             // your application and not part of the Glean API.
             uploadEnabled = settings().isTelemetryEnabled,
+            configuration = Configuration(),
             buildInfo = GleanBuildInfo.buildInfo
         )
     }
@@ -179,6 +183,7 @@ class AppDelegate: UIResponder, UIApplicationDelegate {
             // Here, `Settings` is a method to get user preferences specific to
             // your application, and not part of the Glean API.
             uploadEnabled = Settings.isTelemetryEnabled,
+            configuration = Configuration(),
             buildInfo = GleanMetrics.GleanBuild.info
         )
     }
@@ -202,14 +207,15 @@ The main control for the Glean Python SDK is on the `glean.Glean` singleton.
 
 
 ```python
-from glean import Glean
+from glean import Glean, Configuration
 
 Glean.initialize(
     application_id="my-app-id",
     application_version="0.1.0",
     # Here, `is_telemetry_enabled` is a method to get user preferences specific to
     # your application, and not part of the Glean API.
     upload_enabled=is_telemetry_enabled(),
+    configuration=Configuration(),
 )
 ```
 
@@ -505,4 +511,4 @@ describe("myTestSuite", () => {
 - [Swift API docs](../../../swift/Classes/Glean.html#/s:5GleanAAC10initialize13uploadEnabled13configuration9buildInfoySb_AA13ConfigurationVAA05BuildG0VtF)
 - [Python API docs](../../../python/glean/index.html#glean.Glean.initialize)
 - [Rust API docs](../../../docs/glean/fn.initialize.html)
-- [JavaScript API docs](https://mozilla.github.io/glean.js/functions/core_glean.default.initialize.html)
+- [JavaScript API docs](https://mozilla.github.io/glean.js/getting_started/setup/#initializing-gleanjs)

docs/user/reference/general/shutdown.md

@@ -74,4 +74,3 @@ is explicitly using the `webext` target.
 ## Reference
 
 * [Rust API docs](../../../docs/glean/fn.shutdown.html)
-* [JavaScript API docs](https://mozilla.github.io/glean.js/classes/core_glean.default.html#shutdown)

docs/user/reference/general/toggling-upload-status.md

@@ -131,4 +131,4 @@ uploadSwitch.addEventListener("change", event => {
 * [Swift API docs](../../../swift/Classes/Glean.html#/s:5GleanAAC16setUploadEnabledyySbF)
 * [Python API docs](../../../python/glean/index.html#glean.Glean.set_upload_enabled)
 * [Rust API docs](../../../docs/glean/fn.set_upload_enabled.html)
-* [JavaScript API docs](https://mozilla.github.io/glean.js/classes/core_glean.default.html#setUploadEnabled)
+* [JavaScript API docs](https://mozilla.github.io/glean.js/reference/uploaders/#uploadenabled)

docs/user/reference/metrics/boolean.md

@@ -289,4 +289,3 @@ N/A
 * [Swift API docs](../../../swift/Classes/BooleanMetricType.html)
 * [Python API docs](../../../python/glean/metrics/index.html#glean.metrics.BooleanMetric)
 * [Rust API docs](../../../docs/glean/private/boolean/struct.BooleanMetric.html)
-* [JavaScript API docs](https://mozilla.github.io/glean.js/classes/core_metrics_types_boolean.default.html)

docs/user/reference/metrics/counter.md

@@ -336,4 +336,3 @@ N/A
 * [Swift API docs](../../../swift/Classes/CounterMetricType.html)
 * [Python API docs](../../../python/glean/metrics/index.html#glean.metrics.CounterMetric)
 * [Rust API docs](../../../docs/glean/private/counter/struct.CounterMetric.html)
-* [JavaScript API docs](https://mozilla.github.io/glean.js/classes/core_metrics_types_counter.default.html)

docs/user/reference/metrics/custom_distribution.md

@@ -145,7 +145,7 @@ This API is not currently exposed in Firefox Desktop, see [Bug 1884183](https://
 
 ### `testGetValue`
 
-Gets the recorded value for a given counter metric.  
+Gets the recorded value for a given custom distribution metric.  
 Returns a struct with counts per buckets and total sum if data is stored.  
 Returns a language-specific empty/null value if no data is stored.
 Has an optional argument to specify the name of the ping you wish to retrieve data from, except

docs/user/reference/metrics/datetime.md

@@ -417,4 +417,3 @@ Carefully consider the required resolution for recording your metric, and choose
 * [Swift API docs](../../../swift/Classes/DatetimeMetricType.html)
 * [Python API docs](../../../python/glean/metrics/index.html#glean.metrics.DatetimeMetricType)
 * [Rust API docs](../../../docs/glean/private/struct.DatetimeMetric.html)
-* [Datetime API docs](https://mozilla.github.io/glean.js/classes/core_metrics_types_datetime.default.html)

docs/user/reference/metrics/event.md

@@ -403,4 +403,3 @@ Each extra key contains additional metadata:
 * [Swift API docs](../../../swift/Classes/EventMetricType.html)
 * [Python API docs](../../../python/glean/metrics/index.html#glean.metrics.EventMetricType)
 * [Rust API docs](../../../docs/glean/private/event/struct.EventMetric.html)
-* [JavaScript API docs](https://mozilla.github.io/glean.js/classes/core_metrics_types_event.default.html)