From 29e1f998bab49f895c3f1ba26881bad60433de5f Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 Jul 2020 21:46:36 -0600 Subject: [PATCH 1/5] Proposal to support animated thumbnails --- proposals/2705-thumbnail-requirements.md | 56 ++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 proposals/2705-thumbnail-requirements.md diff --git a/proposals/2705-thumbnail-requirements.md b/proposals/2705-thumbnail-requirements.md new file mode 100644 index 00000000000..6051a92dd71 --- /dev/null +++ b/proposals/2705-thumbnail-requirements.md @@ -0,0 +1,56 @@ +# MSC2705: Animated thumbnails + +Some clients would like animated versions of media for thumbnail purposes. The use cases vary, however +the common ask is frequent enough to warrant an MSC. + +## Proposal + +By specifying `animated=true` in the query string for a `/thumbnail`, the server SHOULD return +an animated thumbnail for the media if possible. When the parameter is `false` the server SHOULD NOT +attempt to generate a thumbnail, as many implementation do today. + +If the server supports the flag, it MUST support animating the following mimetypes: +* `image/gif` +* `image/png` (for animated PNGs - non-animated PNGs cannot be supported) + +Other potential formats include videos and other image types - servers can support additional types +at their discretion. + +If media cannot be animated, a static thumbnail should be returned instead. For example, if the client +requests `?animated=true` on a JPEG the server should not error but instead just return the downsized +JPEG as though the request was `?animated=false`. + +When thumbnailing, servers SHOULD return one of the following types: +* `image/jpeg` (or `image/jpg`) +* `image/png` (non-animated) +* `image/gif` (animated) + +The server's discretion about which one to use is trusted here. Servers can return alternative formats, +however currently it is strongly recommended that servers do not deviate from the set described here. + +The default for `animated` is intentionally left undefined due to a variety of use cases within the +ecosystem which could demand that either `true` or `false` be preferred. Instead of picking a value, +servers SHOULD NOT animate thumbnails by default, but are not required to respect this condition. An +example use case would be a chat platform which is based on Matrix advertising the feature and wanting +to use pre-existing clients. They could go around and fork all the Matrix clients out there, or they +could have their server return a GIFs for everything. + +## Potential issues + +Server load could increase when the server tries to thumbnail a large file. Servers are expected to +mitigate this on their own, such as by providing an option to disable the feature or limiting how/when +they will animate the media. + +## Alternatives + +None that have presented themselves as reasonable. + +## Security considerations + +As mentioned, servers could face resource issues if this feature is left unchecked. + +## Unstable prefix + +While this MSC is not in a released version of the spec, implementations should use `org.matrix.msc2705` +as a prefix. For example, `?org.matrix.msc2705.animated=true`. No unstable endpoints are required due +to backwards compatibility. From 1506eb85c9ad492fe77e996459c8f015b74ab113 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Mon, 27 Jul 2020 21:48:14 -0600 Subject: [PATCH 2/5] add implementation note --- proposals/2705-thumbnail-requirements.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/2705-thumbnail-requirements.md b/proposals/2705-thumbnail-requirements.md index 6051a92dd71..80c57e6ba1e 100644 --- a/proposals/2705-thumbnail-requirements.md +++ b/proposals/2705-thumbnail-requirements.md @@ -54,3 +54,7 @@ As mentioned, servers could face resource issues if this feature is left uncheck While this MSC is not in a released version of the spec, implementations should use `org.matrix.msc2705` as a prefix. For example, `?org.matrix.msc2705.animated=true`. No unstable endpoints are required due to backwards compatibility. + +## Implementations + +This MSC is already implemented by matrix-media-repo. From 3a5936709d9fbebb23d3d7d8aa3d95b4087f97b4 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 Dec 2023 21:49:56 -0700 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Alexey Rusakov --- proposals/2705-thumbnail-requirements.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2705-thumbnail-requirements.md b/proposals/2705-thumbnail-requirements.md index 80c57e6ba1e..4c57eb68971 100644 --- a/proposals/2705-thumbnail-requirements.md +++ b/proposals/2705-thumbnail-requirements.md @@ -7,7 +7,7 @@ the common ask is frequent enough to warrant an MSC. By specifying `animated=true` in the query string for a `/thumbnail`, the server SHOULD return an animated thumbnail for the media if possible. When the parameter is `false` the server SHOULD NOT -attempt to generate a thumbnail, as many implementation do today. +attempt to generate an animated thumbnail, as many implementations do today. If the server supports the flag, it MUST support animating the following mimetypes: * `image/gif` From c07158d984b123c3768d4af85f30e70e243fb280 Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 Dec 2023 22:08:04 -0700 Subject: [PATCH 4/5] Clean up the MSC --- proposals/2705-thumbnail-requirements.md | 70 +++++++++++------------- 1 file changed, 31 insertions(+), 39 deletions(-) diff --git a/proposals/2705-thumbnail-requirements.md b/proposals/2705-thumbnail-requirements.md index 4c57eb68971..05c620513a9 100644 --- a/proposals/2705-thumbnail-requirements.md +++ b/proposals/2705-thumbnail-requirements.md @@ -1,60 +1,52 @@ # MSC2705: Animated thumbnails -Some clients would like animated versions of media for thumbnail purposes. The use cases vary, however -the common ask is frequent enough to warrant an MSC. +Users may already upload animated media to the media repository, such as gifs, webp images, and videos. +When this media is used in an event or avatar, many clients are stuck with a static thumbnail until +the user clicks on it to get the full, unedited, file. Some clients however would prefer to show an +animated thumbnail in certain conditions, like when the user is hovering over the message or avatar. -## Proposal +This proposal introduces a new query parameter to the [`GET /_matrix/media/v3/thumbnail`](https://spec.matrix.org/v1.9/client-server-api/#get_matrixmediav3thumbnailservernamemediaid) +endpoint, allowing clients to specifically request an animated thumbnail. -By specifying `animated=true` in the query string for a `/thumbnail`, the server SHOULD return -an animated thumbnail for the media if possible. When the parameter is `false` the server SHOULD NOT -attempt to generate an animated thumbnail, as many implementations do today. +## Proposal -If the server supports the flag, it MUST support animating the following mimetypes: -* `image/gif` -* `image/png` (for animated PNGs - non-animated PNGs cannot be supported) +A new query parameter, `animated`, is added to the `/thumbnail` endpoint. It has the following behaviour: -Other potential formats include videos and other image types - servers can support additional types -at their discretion. +* When `true`: the server SHOULD return an animated thumbnail if possible/supported. +* When `false`: the server MUST NOT return an animated thumbnail. +* When not provided: the server SHOULD NOT return an animated thumbnail. -If media cannot be animated, a static thumbnail should be returned instead. For example, if the client -requests `?animated=true` on a JPEG the server should not error but instead just return the downsized -JPEG as though the request was `?animated=false`. +The default case is a relaxed version of the `false` behaviour to allow server owners to customize the +default behaviour when their users' clients do not support requesting animated thumbnails. -When thumbnailing, servers SHOULD return one of the following types: -* `image/jpeg` (or `image/jpg`) -* `image/png` (non-animated) -* `image/gif` (animated) +Clients SHOULD respect a user's preference to [reduce motion](https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-reduced-motion) +and request non-animated thumbnails in these cases. -The server's discretion about which one to use is trusted here. Servers can return alternative formats, -however currently it is strongly recommended that servers do not deviate from the set described here. +The content types which are able to be animated is left as an implementation detail. The following +SHOULD be supported at a minimum, however: -The default for `animated` is intentionally left undefined due to a variety of use cases within the -ecosystem which could demand that either `true` or `false` be preferred. Instead of picking a value, -servers SHOULD NOT animate thumbnails by default, but are not required to respect this condition. An -example use case would be a chat platform which is based on Matrix advertising the feature and wanting -to use pre-existing clients. They could go around and fork all the Matrix clients out there, or they -could have their server return a GIFs for everything. +* `image/gif` +* `image/png` ("APNG" format) +* `image/webp` -## Potential issues +The returned content type for an animated thumbnail is additionally left as an implementation detail, +though servers SHOULD use `image/webp` whenever possible. -Server load could increase when the server tries to thumbnail a large file. Servers are expected to -mitigate this on their own, such as by providing an option to disable the feature or limiting how/when -they will animate the media. +When media cannot be animated, such as a PDF or JPEG, the server should return a thumbnail as though +`animated` was `false`. ## Alternatives -None that have presented themselves as reasonable. +No significant alternatives. ## Security considerations -As mentioned, servers could face resource issues if this feature is left unchecked. +Server load could increase when the server tries to thumbnail a large file. Servers are expected to +mitigate this on their own by providing an option to disable the feature or limiting how/when +they will animate the media. ## Unstable prefix -While this MSC is not in a released version of the spec, implementations should use `org.matrix.msc2705` -as a prefix. For example, `?org.matrix.msc2705.animated=true`. No unstable endpoints are required due -to backwards compatibility. - -## Implementations - -This MSC is already implemented by matrix-media-repo. +While this proposal is not considered stable, `animated` is specified as `org.matrix.msc2705.animated` +on requests. No unstable endpoints are required due to backwards compatibility being built-in to the +proposal. From c18ef413677593ac508087e92cedc3dcac44fcaa Mon Sep 17 00:00:00 2001 From: Travis Ralston Date: Fri, 15 Dec 2023 22:10:00 -0700 Subject: [PATCH 5/5] Appease the keepers of mimetypes --- proposals/2705-thumbnail-requirements.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2705-thumbnail-requirements.md b/proposals/2705-thumbnail-requirements.md index 05c620513a9..71108af27e0 100644 --- a/proposals/2705-thumbnail-requirements.md +++ b/proposals/2705-thumbnail-requirements.md @@ -26,7 +26,7 @@ The content types which are able to be animated is left as an implementation det SHOULD be supported at a minimum, however: * `image/gif` -* `image/png` ("APNG" format) +* `image/png` or `image/apng` ("APNG" format) * `image/webp` The returned content type for an animated thumbnail is additionally left as an implementation detail,