Gcode Thumbnail in progress screen

[dmit2k]

Digging through the code I was surprised to discover there is Gcode thumbnail support of some kind.

I would expect seeing Gcode file thumbnail while printing, but I don't see anything like that on the Files Panel as well.

How exactly it should look like? Maybe you could share an example screenshot please?

[suchmememanyskill]

It currently only shows in the file print confirmation screen on klipper. Image needs to be a 32x32 png image.

Will show later

[suchmememanyskill]

@dmit2k

[dmit2k]

@suchmememanyskill

Wow, it is SO tiny? :) I just could not realise it is actual thumbnail! :)

OK, got it, just a follow-up question regarding the GUI customisations - do you use a tool like SquareLine for the GUI or simply hardcore hardcode everything manually?

[suchmememanyskill]

Wow, it is SO tiny? :) I just could not realise it is actual thumbnail! :)

That's 32x32 pixels :)
I couldn't get scaling to work nicely, so i never bothered

OK, got it, just a follow-up question regarding the GUI customisations - do you use a tool like SquareLine for the GUI or simply hardcore hardcode everything manually?

I try to not hardcode pixel offsets etc, newer elements written by me try to use dynamic scaling (i should probably rewrite the gui again now that the backend has been rewritten). But no, i do indeed write everything manually.

[dmit2k]

@suchmememanyskill

IC, I was waiting for the project code to be fully merged with all the features (like backend-frontend separation and serial console) before attempting to replace my stock Neopixel display with a custom one. Now I'm feeling ready to proceed and will definitely do some GUI enhancements. Just a good idea to keep those "inline" with the overall project ideology.

Thank you!

[suchmememanyskill]

@dmit2k

Does something like this look better to you? (Added an option in settings to 2x the image. Although haven't figured out yet how to make it scale it pixel perfect)

1efaa61 i think this is also a decent showing of the old and new style of gui code that i try to write

[dmit2k]

@suchmememanyskill

Does something like this look better to you? (Added an option in settings to 2x the image

Great, it is now much more useable! Thank you!

Although haven't figured out yet how to make it scale it pixel perfect

Just my thoughts:
the CYD-K project now works on different displays and resolutions, are you sure that images (end everything else) should be scaled exactly "X-times"? I would say that we need to scale according to the DPI (which we can calculate) or just the object percentage on the screen.

Another idea could be to use a "helper" app on the host side. It can be implemented at least for the Serial Klipper setup: the "host" daemon can do upscaling and provide necessary higher resolution thumbnails to the display. Although this helper app can also be installed and run for wireless Klipper setups.

1efaa61 i think this is also a decent showing of the old and new style of gui code that i try to write

Nice, will take a look for better understanding!

[suchmememanyskill]

Another idea could be to use a "helper" app on the host side. It can be implemented at least for the Serial Klipper setup: the "host" daemon can do upscaling and provide necessary higher resolution thumbnails to the display. Although this helper app can also be installed and run for wireless Klipper setups.

The issue is that a CYD is very memory constrained (we have ~100kb free!). LVGL at the moment also uses a static buffer for image decoding, and if the image is larger than 32x32 (say 64x64) the buffer is too small to decode it. LVGL also stores it in a 32bit per pixel format, so 16kb is needed just to store the uncompressed image. Which we don't have on some of the larger displays (specifically the 3.5" model, the framebuffer needs to live in ram for that and takes up a large amount of space)

should be scaled exactly "X-times"

Most certainly not, but its a start. I wish to only scale linearly if possible.

[dmit2k]

LVGL also stores it in a 32bit per pixel

OK, now I got your point.. Not played much with LVGL, but is it true also for greyscale images? Maybe we can discard the color / alpha information from the PNG to reduce this overhead?

[suchmememanyskill]

"The whole PNG image is decoded so during decoding RAM equals to image width x image height x 4 bytes are required."

I wish :(

[dmit2k]

And here the helper app can help... We can supply the display a prepared and lightweight image (maybe in some custom format)

What I mean: if we do not care about colours, we can just supply a bitmap matrix. Anyway, this thumbnail will not be animated or transformed any further - it should be statically present on the single "Printing" panel at specific position

[suchmememanyskill]

But with a helper tool it won't work for wifi, and it's extra setup i wish to avoid at all costs

[dmit2k]

@suchmememanyskill

here are some suggestions from ChatGPT:

Optimizing PNG Decoding on ESP32 S3 with LVGL

If you're facing memory constraints on an ESP32 S3 when decoding and displaying PNG images using LVGL, here are several strategies to optimize and resolve the issue. This is particularly relevant for projects like CYD Klipper where memory is very limited (~100kB free).

---

Problem Description

The key challenges include:

1. LVGL uses a static buffer for image decoding, which is often too small for larger images (e.g., 64x64).
2. PNG images require a significant amount of memory during decoding (width x height x 4 bytes).
3. Larger displays (e.g., 3.5-inch) require a substantial framebuffer, consuming available RAM.

Goal: Decode a PNG thumbnail (32x32, received from the printer) and display it on a higher-resolution screen with minimal memory usage.

---

Solutions

1. Reduce Color Depth

Lower the image storage format from 32 bits per pixel to 16 or 8 bits:

• LVGL supports 16-bit (RGB565) or 8-bit indexed color formats.
• This reduces memory requirements significantly:
  • For a 64x64 image in RGB565, only 8kB is required (64x64x2 bytes).

LVGL Configuration:

#define LV_COLOR_DEPTH 16  // Or 8 for Indexed 8-bit

---

2. Decode PNG Incrementally

Use partial decoding to reduce memory usage:

• Implement a custom decoder that reads the PNG file line-by-line or in small blocks, rendering directly to the display.
• Use a library like lodepng for incremental decoding.

Example (Pseudo-code):

uint8_t line_buffer[DISPLAY_WIDTH * LV_COLOR_DEPTH / 8];  // Buffer for a single line
for (int y = 0; y < image_height; y++) {
    decode_png_line(png_data, line_buffer, y);
    draw_line_on_display(x, y, line_buffer);
}

---

3. Use JPEG Instead of PNG

If PNG is too memory-intensive, consider converting thumbnails on the printer side to JPEG or BMP:

• JPEG images use less memory due to lossy compression.
• ESP32 supports JPEG decoding with libraries like ESP32 JPEG Decoder.

On the Printer (Klipper):
Use Python to convert PNG to JPEG:

from PIL import Image
img = Image.open("thumbnail.png")
img.save("thumbnail.jpg", "JPEG", quality=85)

---

4. Upscale Low-Resolution Images

If the thumbnail remains small (32x32), use LVGL’s built-in scaling to display it at a higher resolution:

• LVGL supports image scaling:

lv_img_set_zoom(img, scale_factor);

For example, to upscale 32x32 to 128x128, use scale_factor = 400.

---

5. Leverage PSRAM

If your ESP32 S3 includes external PSRAM:

• Move framebuffers and temporary buffers to PSRAM to free internal RAM:

#define CONFIG_SPIRAM_USE_CAPS_ALLOC 1
#define CONFIG_SPIRAM_SUPPORT 1

• Configure LVGL to use PSRAM:

#define LV_MEM_ADR (void *)0x3F800000  // PSRAM address
#define LV_MEM_SIZE (8 * 1024 * 1024)  // Size of PSRAM

---

6. Preconvert Images on the Printer

If decoding PNGs is too complex, convert the image to a simpler format (e.g., RAW RGB565) on the printer side:

• On the printer (e.g., Klipper):

img = Image.open("thumbnail.png").convert("RGB").resize((128, 128))
img.save("thumbnail.raw", format="RAW")

• On the ESP32, load the raw buffer into memory and display it directly.

---

Summary

To optimize PNG decoding on ESP32 S3 for CYD Klipper:

1. Use RGB565 color depth to reduce memory usage.
2. Implement partial decoding or convert images to JPEG.
3. Upscale smaller images directly on the display.
4. Utilize PSRAM if available for framebuffers and buffers.
5. Preconvert images to simpler formats (e.g., RAW) on the printer side.

Let me know if you need code examples or deeper integration tips!

[dmit2k]

for me PSRAM and reducing color depth are the most promising solutions...