Real-time update of image control breaks socket connection

[cansik]

Duplicate Check

• I have searched the opened issues and there are no duplicates

Describe the bug

If you try to update the image source (src_base64) of an Image control too many times from a different thread, the update won't work properly and the socket connection may even break. The image is only updated from time to time because too many updates are sent to the component. If the update calls are limited (not every 2-5ms) it seems to work.

In my opinion this could be a locking problem with the update method or a buffer problem.

The error occurred when implementing a live webcam view using opencv-python. If there is a better solution for displaying a stream of realtime processed numpy arrays, I would be happy to know about it as well.

Code sample

Code

import base64
import threading
import time

import cv2
import flet as ft
import numpy as np


def convert_image(image_array: np.ndarray) -> str:
    _, encoded_image = cv2.imencode(f".ppm", image_array)
    return base64.b64encode(encoded_image).decode('utf-8')


def main(page: ft.Page):
    page.add(ft.Text("Update Issue"))

    text = ft.Text("time")
    page.add(text)

    image_control = ft.Image(
        src=f"",
        width=640,
        height=480,
        fit=ft.ImageFit.CONTAIN,
    )
    page.add(image_control)

    def update():
        # pre-generate the images
        images = [(np.random.rand(480, 640, 3) * 255).astype(np.uint8)
                  for _ in range(0, 10)]
        image_index = 0

        while True:
            # update text
            text.value = f"{time.time() * 1000}"
            page.update(text)

            # update image
            start = time.time()
            image_index = (image_index + 1) % len(images)
            image = images[image_index]
            data = convert_image(image)
            end = time.time()

            print(f"Encoding Time: {(end - start) * 1000:.2f}ms")

            image_control.src_base64 = data
            image_control.update()

            # adding this keeps it running forever
            # time.sleep(0.03)

    t = threading.Thread(target=update, daemon=True)
    t.start()


ft.app(main)

To reproduce

Please install numpy and opencv-python as dependencies.

pip install numpy opencv-python

Run the example and see that the image is not updated correctly (one update per update call). You can uncomment the time.sleep call and see that it works better with a bit of sleep.

Expected behavior

The frames are either displayed or skipped, but not been shown delayed.

Screenshots / Videos

Captures

image-control-real-time.mp4

Without time.sleep.

image-control-with-sleep.mov

With time.sleep enabled.

Operating System

macOS

Operating system details

MacOS 15.0.1

Flet version

0.24.1

Regression

I'm not sure / I don't know

Suggestions

As mentioned above, it appears that the page.update() method can be called multiple times, but internally passes the work to another thread. This causes either a concurrency problem or a buffer fill.

Logs

Logs

Encoding Time: 2.10ms
Encoding Time: 1.91ms
Encoding Time: 1.81ms
Encoding Time: 1.55ms
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
Encoding Time: 1.91ms
Encoding Time: 1.61ms
Encoding Time: 1.60ms
Encoding Time: 1.78ms
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
Encoding Time: 1.79ms
Encoding Time: 1.77ms
Encoding Time: 1.98ms
Encoding Time: 1.63ms
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
Encoding Time: 1.73ms

Additional details

No response