TW-1049 Notification improve iOS notification

docs/adr/0012-improve-ios-notification.md

@@ -0,0 +1,73 @@
+# 12. Improve iOS notification
+
+Date: 2023-12-06
+
+## Status
+
+Accepted
+
+## Context
+The motivation behind this decision is the inability to execute Dart code for decrypting notifications when the iOS app is running in the background. This limitation is different from Android or Web, where Dart code can easily run in such scenarios.
+
+## Decision
+We have decided to make the following changes to address the mentioned issue:
+
+1. **Notification Service Extension (NSE):**
+   - Utilize the [Notification Service Extension](https://developer.apple.com/documentation/usernotifications/modifying_content_in_newly_delivered_notifications) provided by Apple to modify the content of notifications.
+
+2. **MatrixRustSDK Integration:**
+   - Integrate MatrixRustSDK to decrypt messages within the Notification Service Extension. The MatrixRustSDK version and other dependencies should be synchronized with Element X to avoid unexpected errors. Reference the synchronization from [Element X project.yml](https://github.com/vector-im/element-x-ios/blob/main/project.yml#L46).
+
+3. **Automated Script for Source Code Copy:**
+   - Develop a script written in Node.js to automate the process of copying the entire source code related to the Notification Service Extension from Element X, a known version. Detail at [scripts/copy-nse/README.MD](../../scripts/copy-nse/README.MD)
+
+4. **Dart Code Adjustment:**
+   - Modify Dart code to ensure compatibility with the Notification Service Extension from Element X.
+
+5. **Debugging Patch with AppDelegate:**
+   - Add the patch `scripts/patchs/ios-extension-debug.patch` to address debugging issues. This patch removes FlutterAppDelegate and suggests using AppDelegate instead for smoother debugging during Notification Service Extension development.
+
+6. **Add pusher_notification_client_identifier to Default Payload:**
+   - Extend the default notification payload by adding the `pusher_notification_client_identifier` field. This supports multi-account scenarios, where each notification will have a `client_identifier` to identify the user for decryption. For now it is SHA256 of userId
+
+7. **Keychain Access Groups:**
+   - Utilize `keychain-access-groups` to enable data exchange between the Notification Service Extension (NSE) and the main app by using a shared Keychain Access Group. This ensures security and safe data transmission (See details at [link](https://developer.apple.com/documentation/security/keychain_services/keychain_items/sharing_access_to_keychain_items_among_a_collection_of_apps/)).
+
+8. **INSendMessageIntent:**
+   - Use `INSendMessageIntent` to support displaying profile pictures in notifications. Instead of using regular notifications, integrating `INSendMessageIntent` ensures that the sender's image is displayed correctly in the notification (See details at [link](https://stackoverflow.com/questions/68198452/ios-15-communication-notification-picture-not-shown)).
+
+9. **iOS Version Support Below 16:**
+   - Communicate that Element X's Notification Service Extension supports iOS 16 and above. Therefore, Twake NSE will also have a similar requirement. Users with iOS versions below 16 can still install Twake, but they won't be able to decrypt notifications.
+
+## Consequences
+Implementing these decisions has the following consequences:
+
+1. **Notification Discrepancy:**
+   - Notifications will behave differently in the background and foreground because they are handled by Swift in the background and Dart in the foreground.
+
+2. **Integration Challenges:**
+   - There might be challenges in seamlessly integrating Notification Service Extension with MatrixRustSDK and adjusting it to fit the current source code. The MatrixRustSDK version and dependencies should be synchronized with Element X to avoid unexpected errors.
+
+3. **Automated Source Code Copy:**
+   - Introducing an automated Node.js script streamlines the process of copying the source code from Element X, enhancing efficiency and reducing manual errors during this task.
+
+4. **Message Decryption Research:**
+   - Further research is required to successfully decrypt encrypted messages. This introduces a potential risk as it may involve understanding specific nuances when integrating MatrixRustSDK with Notification Service Extension.
+
+5. **Maintenance Overhead:**
+   - Copying source code from Element X may create maintenance overhead, as any updates or changes in Element X's source code need manual integration and synchronization.
+
+6. **Improved Debugging:**
+   - The debugging patch aims to optimize the debugging experience by addressing issues related to FlutterAppDelegate in the context of Notification Service Extension.
+
+7. **Multi-Account Support:**
+   - Multi-account support is achieved by adding `pusher_notification_client_identifier` to the default notification payload.
+
+8. **Security with Keychain Access Groups:**
+   - Ensures secure data transmission between Notification Service Extension and the main app using Keychain Access Groups.
+
+9. **Profile Picture Display Enhancement:**
+   - The use of `INSendMessageIntent` enhances the user experience by ensuring the correct display of sender profile pictures in notifications.
+
+10. **iOS 16 Requirement:**
+    - Communicates that Element X's Notification Service Extension requires iOS 16 or later, and Twake NSE has a similar requirement. Users below iOS 16 can install Twake but won't decrypt notifications.

ios/NSE/AttributedString.swift

@@ -0,0 +1,83 @@
+//
+// Copyright 2023 New Vector Ltd
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+import Foundation
+
+extension AttributedString {
+    var formattedComponents: [AttributedStringBuilderComponent] {
+        runs[\.blockquote].map { value, range in
+            var attributedString = AttributedString(self[range])
+
+            // Remove trailing new lines if any
+            if attributedString.characters.last?.isNewline ?? false,
+               let range = attributedString.range(of: "\n", options: .backwards, locale: nil) {
+                attributedString.removeSubrange(range)
+            }
+
+            let isBlockquote = value != nil
+
+            return AttributedStringBuilderComponent(attributedString: attributedString, isBlockquote: isBlockquote)
+        }
+    }
+
+    /// Replaces the specified placeholder with a string that links to the specified URL.
+    /// - Parameters:
+    ///   - linkPlaceholder: The text in the string that will be replaced. Make sure this is unique within the string.
+    ///   - string: The text for the link that will be substituted into the placeholder.
+    ///   - url: The URL that the link should open.
+    mutating func replace(_ linkPlaceholder: String, with string: String, asLinkTo url: URL) {
+        // Replace the placeholder with a link.
+        var replacement = AttributedString(string)
+        replacement.link = url
+        replace(linkPlaceholder, with: replacement)
+    }
+
+    /// Replaces the specified placeholder with the supplied attributed string.
+    /// - Parameters:
+    ///   - placeholder: The text in the string that will be replaced. Make sure this is unique within the string.
+    ///   - attributedString: The text for the link that will be substituted into the placeholder.
+    mutating func replace(_ placeholder: String, with replacement: AttributedString) {
+        guard let range = range(of: placeholder) else {
+            MXLog.failure("Failed to find the placeholder to be replaced.")
+            return
+        }
+
+        // Replace the placeholder.
+        replaceSubrange(range, with: replacement)
+    }
+
+    /// Returns a new attributed string, created by replacing any hard coded `UIFont` with
+    /// a simple presentation intent. This allows simple formatting to respond to Dynamic Type.
+    ///
+    /// Currently only supports regular and bold weights.
+    func replacingFontWithPresentationIntent() -> AttributedString {
+        var newValue = self
+        for run in newValue.runs {
+            guard let font = run.uiKit.font else { continue }
+            newValue[run.range].inlinePresentationIntent = font.fontDescriptor.symbolicTraits.contains(.traitBold) ? .stronglyEmphasized : nil
+            newValue[run.range].uiKit.font = nil
+        }
+        return newValue
+    }
+
+    /// Makes the entire string bold by setting the presentation intent to strongly emphasized.
+    ///
+    /// In practice, this is rendered as semibold for smaller font sizes and just so happens to nicely
+    /// line up with the semibold → bold font switch used by compound.
+    mutating func bold() {
+        self[startIndex..<endIndex].inlinePresentationIntent = .stronglyEmphasized
+    }
+}

ios/NSE/AvatarSize.swift

@@ -0,0 +1,110 @@
+//
+// Copyright 2022 New Vector Ltd
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+import Foundation
+import UIKit
+
+enum AvatarSize {
+    case user(on: UserAvatarSizeOnScreen)
+    case room(on: RoomAvatarSizeOnScreen)
+    //  custom
+    case custom(CGFloat)
+
+    /// Value in UIKit points
+    var value: CGFloat {
+        switch self {
+        case .user(let screen):
+            return screen.value
+        case .room(let screen):
+            return screen.value
+        case .custom(let val):
+            return val
+        }
+    }
+
+    /// Value in pixels by using the scale of the main screen
+    var scaledValue: CGFloat {
+        value * UIScreen.main.scale
+    }
+}
+
+enum UserAvatarSizeOnScreen {
+    case timeline
+    case home
+    case settings
+    case roomDetails
+    case startChat
+    case memberDetails
+    case inviteUsers
+    case readReceipt
+    case editUserDetails
+
+    var value: CGFloat {
+        switch self {
+        case .readReceipt:
+            return 16
+        case .timeline:
+            return 32
+        case .home:
+            return 32
+        case .settings:
+            return 52
+        case .roomDetails:
+            return 44
+        case .startChat:
+            return 36
+        case .memberDetails:
+            return 70
+        case .inviteUsers:
+            return 56
+        case .editUserDetails:
+            return 96
+        }
+    }
+}
+
+enum RoomAvatarSizeOnScreen {
+    case timeline
+    case home
+    case messageForwarding
+    case details
+    case notificationSettings
+
+    var value: CGFloat {
+        switch self {
+        case .notificationSettings:
+            return 30
+        case .timeline:
+            return 32
+        case .messageForwarding:
+            return 36
+        case .home:
+            return 52
+        case .details:
+            return 70
+        }
+    }
+}
+
+extension AvatarSize {
+    var size: CGSize {
+        CGSize(width: value, height: value)
+    }
+
+    var scaledSize: CGSize {
+        CGSize(width: scaledValue, height: scaledValue)
+    }
+}

ios/NSE/BackgroundTaskProtocol.swift

@@ -0,0 +1,43 @@
+//
+// Copyright 2022 New Vector Ltd
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+import Foundation
+
+typealias BackgroundTaskExpirationHandler = (BackgroundTaskProtocol) -> Void
+
+/// BackgroundTaskProtocol is the protocol describing a background task regardless of the platform used.
+protocol BackgroundTaskProtocol: AnyObject {
+    /// Name of the background task for debug.
+    var name: String { get }
+
+    /// `true` if the background task is currently running.
+    var isRunning: Bool { get }
+
+    /// Flag indicating the background task is reusable. If reusable, `name` is the key to distinguish background tasks.
+    var isReusable: Bool { get }
+
+    /// Elapsed time after the task started. In milliseconds.
+    var elapsedTime: TimeInterval { get }
+
+    /// Expiration handler for the background task
+    var expirationHandler: BackgroundTaskExpirationHandler? { get }
+
+    /// Method to be called when a task reused one more time. Should only be valid for reusable tasks.
+    func reuse()
+
+    /// Stop the background task. Cannot be started anymore. For reusable tasks, should be called same number of times `reuse` called.
+    func stop()
+}

ios/NSE/BackgroundTaskServiceProtocol.swift

@@ -0,0 +1,45 @@
+//
+// Copyright 2022 New Vector Ltd
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+import Foundation
+
+@MainActor
+protocol BackgroundTaskServiceProtocol {
+    func startBackgroundTask(withName name: String,
+                             isReusable: Bool,
+                             expirationHandler: (() -> Void)?) -> BackgroundTaskProtocol?
+}
+
+extension BackgroundTaskServiceProtocol {
+    func startBackgroundTask(withName name: String) -> BackgroundTaskProtocol? {
+        startBackgroundTask(withName: name,
+                            expirationHandler: nil)
+    }
+
+    func startBackgroundTask(withName name: String,
+                             isReusable: Bool) -> BackgroundTaskProtocol? {
+        startBackgroundTask(withName: name,
+                            isReusable: isReusable,
+                            expirationHandler: nil)
+    }
+
+    func startBackgroundTask(withName name: String,
+                             expirationHandler: (() -> Void)?) -> BackgroundTaskProtocol? {
+        startBackgroundTask(withName: name,
+                            isReusable: false,
+                            expirationHandler: expirationHandler)
+    }
+}