[Bug]: Serious Photo Quality degradation on each edit.

[isenbj]

Package Version

4.3.0

Flutter Version

3.22.2

Platforms

iOS

How to reproduce?

In my app I use photos that are about 480x640. I've noticed that the pro-image-editor seriously degrades the quality of the image upon saving. This is not noticeable on high quality images unless many saves are done. I've attached a complete sample app, along with the photos that demonstrate the issue. Note the image_compress library is not needed, but the issue on higher dimensioned images shows much slower and needs dozens of edits to notice.

I have tried different configs too like so:

ImageGeneratioConfigs(
              pngLevel: 0,
              outputFormat: OutputFormat.jpg,
              jpegQuality: 100,
            )

Logs (optional)

No response

Example code (optional)

Expand Code

dart
import 'dart:typed_data';
import 'package:flutter/material.dart';
import 'package:image_picker/image_picker.dart';
import 'package:flutter_image_compress/flutter_image_compress.dart';
import 'package:pro_image_editor/models/editor_callbacks/pro_image_editor_callbacks.dart';
import 'package:pro_image_editor/models/editor_configs/pro_image_editor_configs.dart';
import 'package:pro_image_editor/modules/main_editor/main_editor.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Image Picker Example',
      theme: ThemeData(primarySwatch: Colors.blue),
      home: const ImagePickerScreen(),
    );
  }
}

class ImagePickerScreen extends StatefulWidget {
  const ImagePickerScreen({super.key});

  @override
  State<ImagePickerScreen> createState() => _ImagePickerScreenState();
}

class _ImagePickerScreenState extends State<ImagePickerScreen> {
  Uint8List? _imageBytes;
  final ImagePicker _picker = ImagePicker();

  Future<void> _pickImage() async {
    final pickedFile = await _picker.pickImage(source: ImageSource.gallery);
    if (pickedFile != null) {
      Uint8List originalBytes = await pickedFile.readAsBytes();
      Uint8List compressedBytes = await FlutterImageCompress.compressWithList(
        originalBytes,
        quality: 100,
        format: CompressFormat.png,
        minWidth: 480,
        minHeight: 640,
      );

      setState(() => _imageBytes = compressedBytes);
    }
  }

  void _openEditor(BuildContext context) async {
    Navigator.push(
      context,
      MaterialPageRoute(
        builder: (context) => ProImageEditor.memory(
          _imageBytes!,
          configs: const ProImageEditorConfigs(
            imageEditorTheme: ImageEditorTheme(
              layerInteraction: ThemeLayerInteraction(
                buttonRadius: 10,
                strokeWidth: 1.2,
                borderElementWidth: 7,
                borderElementSpace: 5,
                borderColor: Colors.blue,
                removeCursor: SystemMouseCursors.click,
                rotateScaleCursor: SystemMouseCursors.click,
                editCursor: SystemMouseCursors.click,
                hoverCursor: SystemMouseCursors.move,
                borderStyle: LayerInteractionBorderStyle.solid,
                showTooltips: false,
              ),
            ),
            layerInteraction: LayerInteraction(selectable: LayerInteractionSelectable.enabled, initialSelected: true),
            paintEditorConfigs: PaintEditorConfigs(canToggleFill: false),
            helperLines: HelperLines(hitVibration: false),
            blurEditorConfigs: BlurEditorConfigs(enabled: false),
            emojiEditorConfigs: EmojiEditorConfigs(enabled: false),
          ),
          callbacks: ProImageEditorCallbacks(
            onImageEditingComplete: (Uint8List bytes) async {
              setState(() => _imageBytes = bytes);
              Navigator.pop(context);
            },
          ),
        ),
      ),
    );
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Image Picker Example'),
      ),
      body: Center(child: _imageBytes != null ? Image.memory(_imageBytes!) : const Text('No image selected.')),
      floatingActionButton: _imageBytes == null
          ? null
          : FloatingActionButton(
              onPressed: () => _openEditor(context),
              tooltip: 'Edit Image',
              child: const Icon(Icons.edit),
            ),
      bottomNavigationBar: Padding(
        padding: const EdgeInsets.all(16.0),
        child: ElevatedButton(onPressed: _pickImage, child: const Text('Pick Image from Gallery')),
      ),
    );
  }
}

  image_picker: ^1.0.7
  flutter_image_compress: ^2.2.0
  pro_image_editor: ^4.3.0

Device Model (optional)

iPhone 15 Pro

[isenbj]

I've even tried on the demo provided doing an edit even with no changes degrades the quality. This is from the default editor option.

I can't say I understand the code enough to change it. There are too many branches in the short time I have to dive deep into it. But it seems that this issue comes from the fact a screenshot is taken of the original image. This is flawed in that different screen sizes have different pixel densities, and flutter may behave any which way when taking this screenshot.

I think the original image Uint8List of bytes should be preserved edit-to-edit. Then the changes can be added back into the image pixel by pixel. This won't be as fast as a screenshot, but preserves quality, and at least a good configurable parameter option added to the package. For multi-process capable devices, as new objects are drawn onto the image they can be sent to a background isolate with the objects relative position and overwrite the bytes of the original with the new. This is just one example of how it could be done.

I also had the idea of on capture, the background image is removed, and replaced with a color that is unlikely to appear in either the original photo, or the edited layers. like RGB(1,2,3). Then a screenshot happens which will capture all of the edits with this unique background. Then pixel by pixel each one that is not RGB(1,2,3) gets overlayed onto the bytes of the original.

In both cases, the only pixels changing would be those that actually changed.

[hm21]

Thank you for reporting that issue with all details.

To create the final image, the image editor captures each rendered pixel. The editor actually respects the pixel ratio. As an example, you have an image with a width of 2000px but only a screen with 500px you will have a pixel ratio of 4. That means when the editor generates the final image it will render it with a 4 times bigger size that we still reach our 2000px width. The issue sounds to me like there is a problem with the progress to setting the pixel ratio correctly.

If we didn't set the pixel ratio manually, the editor will calculate it automatically here and apply it here.

I'm not sure if there is a mistake how I calculate the pixel ratio or the way how flutter use it then. The other possible issue can possibly be that the image formats JPEG and PNG always reduce the quality in the case the package image has an issue.
Anyway, just to make sure the issue is really on the pixel ratio, you can check if it reduces the output width. If the output width is not same as the input width, is the issue that the pixel-ratio is incorrect.
If the output width is the same, you can try all image formats, not only JPEG and PNG. Keep in mind that some formats may not display in Flutter depending on your device.

I think the original image Uint8List of bytes should be preserved edit-to-edit. Then the changes can be added back into the image pixel by pixel...

Technically, it's possible, but it sounds complicated, and I think there's a big performance impact even if you write native code. However, if you want to create a pull request, I will review and compare with the current way it captures.

FYI in case you would like to use the original image and just apply all layers and crop transformations, I recommend you to check out the export/import example here. It allows you to export everything as JSON which you can later directly import.

[isenbj]

I'm not sure if there is a mistake how I calculate the pixel ratio or the way how flutter use it then.

I don't think you are calculating it incorrectly based on my observation here (on iPad):

but perhaps the error is in the usage of this ratio on different device sizes. I've made a few more observations that may help get to the bottom of it:

1. Behavior is different on an iPad vs an iPhone doing the exact same thing. An iPad may take many more edits to show the artifacts where the phone shows it right away. This is hinting that the screenshot may be screen size dependent and introduce these issues. I know an iPhone has a higher pixel density, and flutter does some conversions here with virtual pixel widths and such.
2. captureOnlyBackgroundImageArea seems to have an effect here as well. See these three series of images (on iPhone):

Drawing a circle on the image, looks fine before any saving is done.

Save is pressed, and I create a dialog with a photo at this point in code:

      ui.Image image = await boundary.toImage(pixelRatio: pixelRatio);

      await showDialog(
        context: context!,
        builder: (ctx) => Dialog.fullscreen(
          child: Scaffold(
            appBar: AppBar(),
            body: Center(child: RawImage(image: image)),
          ),
        ),
      );

      if (_configs.imageGenerationConfigs.captureOnlyBackgroundImageArea) {

This photo was captured just before saving the image in the condition based upon the background image area:

        image = await recorder
            .endRecording()
            .toImage(cropWidth.ceil(), cropHeight.ceil());

        await showDialog(
          context: context!,
          builder: (ctx) => Dialog.fullscreen(
            child: Scaffold(
              appBar: AppBar(),
              body: Center(child: RawImage(image: image)),
            ),
          ),
        );

From these photos you can see there is something fishy going on here with capturing the image when `captureOnlyBackgroundImageArea` is true.

I don't think it is necessary to overwrite the pixel values directly to preserve the bytes, since this is computationally intensive, unless we can get to the bottom of why it is happening in the first place to prove it is not necessary.

[isenbj]

Just as a note, there still is quality degradation when captureOnlyBackgroundImageArea = false but it comes much slower, and is less dramatic. Here is a photo after 11 edits.

The best user experience is of course a consistent image quality through edits. The nature of my app is such that they may have up to 20 edits on a photo throughout its lifespan from different people annotating it, and adding details.

I can have workarounds for this where I just record the history and reapply it every time a user goes to edit from the original, but this is a bandaid solution I think from the intended workings of the package. And I would need to store multiple different pieces of information upping my storage costs per photo (which there are hundreds of photos, hence the 480x640 resolution I have). Those being the original photo, the history of edits, and the latest image with edits applied for thumbnails that gets overridden each update.

As you can imagine, I don't want to inherit these extra storage costs.

[hm21]

Okay, that's interesting information. I also played around with your code example and edited it a bit for some tests. I can now say that the part that makes the problem is between there we read the renderObject, and we convert it to raw RGB pixels. The other stuff like decoders and this stuff I tested, and it didn't have any negative effect. In my tests I also found out that if I set the pixelRatio higher than required the result is much better, but still not perfect. The image also changed the x position slowly in my tests.

Anyway, when I find time again I will do some more tests, but currently I'm very limited with the time I can invest.
In my opinion, the issue may now be the way we read it, or maybe the boxfit of the image has a negative effect. The other is that flutter doesn't return the pixels correctly. However, below I post the edited example code from you which allows you to test it a bit faster, and you can also test it with and without the editor and also toggle between the original and the edited image.

As you can imagine, I don't want to inherit these extra storage costs.

Yes, I absolutely agree in case you need to edit multiple times, but it's not necessary to undo the edited stuff from the user before, it makes no sense if you also save the state history. FYI for the case that in the future your users also want to have the possibility to undo changes from before, and you will export/import the state history, it might also be interesting for you that the editor also allows to just generate a thumbnail with your specific size. You can see an example here.

I don't think it is necessary to overwrite the pixel values directly to preserve the bytes...

I didn't test it now, but if I recall it correctly had this code a small performance impact from around 30ms. Technically, it's possible to do it inside isolate and web-worker, but I didn't have time to change it and because of the small impact it is on my priority list not on a high position.
For the case you want to disable this option but still just cut the image area, you can also set captureOnlyDrawingBounds to true. This will check in the separated thread where are the image bounding. Keep in mind that way will consume a lot more performance because in the end there is a for loop which starts to read the pixels which one is transparent and which one not and to be honest, dart is very slow at reading many pixels compared to other languages.

Expand Code

import 'dart:math';
import 'dart:typed_data';
import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';
import 'package:image_picker/image_picker.dart';
import 'package:pro_image_editor/pro_image_editor.dart';
import 'dart:ui' as ui;

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Image Picker Example',
      theme: ThemeData(primarySwatch: Colors.blue),
      debugShowCheckedModeBanner: false,
      home: const ImagePickerScreen(),
    );
  }
}

class ImagePickerScreen extends StatefulWidget {
  const ImagePickerScreen({super.key});

  @override
  State<ImagePickerScreen> createState() => _ImagePickerScreenState();
}

class _ImagePickerScreenState extends State<ImagePickerScreen> {
  bool _captureWithoutEditor = true;
  bool _showOriginal = false;
  Uint8List? _imageBytes;
  Uint8List? _originalBytes;
  final ImagePicker _picker = ImagePicker();
  int _testCount = 0;

  Future<void> _pickImage() async {
    final pickedFile = await _picker.pickImage(source: ImageSource.gallery);
    if (pickedFile != null) {
      _originalBytes = await pickedFile.readAsBytes();

      setState(() => _imageBytes = _originalBytes);
    }
  }

  void _openEditor() async {
    _showOriginal = false;

    final demoRecorderKey = GlobalKey();
    final editor = GlobalKey<ProImageEditorState>();
    Size screenSize = Size.zero;

    double radius = 140;
    double angle = (_testCount * 2 * pi) / 10;
    double x = radius * cos(angle);
    double y = radius * sin(angle);

    if (_captureWithoutEditor) {
      Future.delayed(const Duration(milliseconds: 100), () async {
        /*   var infos =
            await decodeImageInfos(bytes: _imageBytes!, screenSize: screenSize); */

        final RenderRepaintBoundary boundary = demoRecorderKey.currentContext!
            .findRenderObject()! as RenderRepaintBoundary;

        final ui.Image image = await boundary.toImage(pixelRatio: 4);

        final ByteData? byteData =
            await image.toByteData(format: ui.ImageByteFormat.png);

        _testCount++;
        setState(() => _imageBytes = byteData!.buffer.asUint8List());

        var decodedImage = await decodeImageFromList(_imageBytes!);
        print(
          Size(
            decodedImage.width.toDouble(),
            decodedImage.height.toDouble(),
          ),
        );

        if (mounted) Navigator.pop(context);
      });
    }

    Navigator.push(
      context,
      PageRouteBuilder(
        transitionDuration: Duration.zero,
        reverseTransitionDuration: Duration.zero,
        pageBuilder: (context, animation1, animation2) => _captureWithoutEditor
            ? Scaffold(
                backgroundColor: Colors.black,
                body: LayoutBuilder(builder: (context, constraints) {
                  screenSize = constraints.biggest;
                  return RepaintBoundary(
                    key: demoRecorderKey,
                    child: Stack(
                      alignment: Alignment.center,
                      children: [
                        Center(
                          child: Image.memory(
                            _imageBytes!,
                          ),
                        ),
                        Positioned(
                          top: screenSize.height / 2 + y,
                          left: screenSize.width / 2 + x,
                          child: FractionalTranslation(
                            translation: const Offset(-0.5, -0.5),
                            child: Container(
                              decoration: BoxDecoration(
                                color: Colors.white,
                                borderRadius: BorderRadius.circular(7),
                              ),
                              padding: const EdgeInsets.symmetric(
                                horizontal: 7,
                                vertical: 3,
                              ),
                              child: Text(
                                '$_testCount',
                                overflow: TextOverflow.ellipsis,
                                style: const TextStyle(
                                  color: Colors.red,
                                  fontSize: 24,
                                  fontWeight: FontWeight.w500,
                                ),
                              ),
                            ),
                          ),
                        )
                      ],
                    ),
                  );
                }),
              )
            : ProImageEditor.memory(
                _imageBytes!,
                key: editor,
                configs: const ProImageEditorConfigs(
                  imageEditorTheme: ImageEditorTheme(
                    layerInteraction: ThemeLayerInteraction(
                      buttonRadius: 10,
                      strokeWidth: 1.2,
                      borderElementWidth: 7,
                      borderElementSpace: 5,
                      borderColor: Colors.blue,
                      removeCursor: SystemMouseCursors.click,
                      rotateScaleCursor: SystemMouseCursors.click,
                      editCursor: SystemMouseCursors.click,
                      hoverCursor: SystemMouseCursors.move,
                      borderStyle: LayerInteractionBorderStyle.solid,
                      showTooltips: false,
                    ),
                  ),
                  layerInteraction: LayerInteraction(
                    selectable: LayerInteractionSelectable.disabled,
                    initialSelected: false,
                  ),
                  paintEditorConfigs: PaintEditorConfigs(canToggleFill: false),
                  helperLines: HelperLines(hitVibration: false),
                  blurEditorConfigs: BlurEditorConfigs(enabled: false),
                  emojiEditorConfigs: EmojiEditorConfigs(enabled: false),
                  imageGenerationConfigs: ImageGeneratioConfigs(
                    allowEmptyEditCompletion: true,
                    captureOnlyBackgroundImageArea: false,
                    captureOnlyDrawingBounds: true,
                    // customPixelRatio: 3,
                  ),
                ),
                callbacks: ProImageEditorCallbacks(
                  mainEditorCallbacks: MainEditorCallbacks(
                    onAfterViewInit: () async {
                      await Future.delayed(const Duration(milliseconds: 500));
                      editor.currentState!.addLayer(
                        TextLayerData(
                          text: '$_testCount',
                          color: Colors.red,
                          customSecondaryColor: true,
                          background: Colors.white,
                          fontScale: 1.6,
                          offset: Offset(x, y),
                        ),
                      );
                      await Future.delayed(const Duration(milliseconds: 1));
                      editor.currentState!.doneEditing();
                    },
                  ),
                  onImageEditingComplete: (Uint8List bytes) async {
                    if (bytes.isNotEmpty) {
                      _testCount++;
                      var decodedImage = await decodeImageFromList(bytes);
                      print(
                        Size(
                          decodedImage.width.toDouble(),
                          decodedImage.height.toDouble(),
                        ),
                      );
                      setState(() => _imageBytes = bytes);
                    }

                    if (context.mounted) Navigator.pop(context);
                  },
                ),
              ),
      ),
    ).whenComplete(() async {
      if (_testCount % 10 != 0) {
        _openEditor();
      }
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Image Picker Example'),
        actions: [
          IconButton(
            onPressed: () => setState(() {
              _showOriginal = !_showOriginal;
            }),
            tooltip: _showOriginal ? 'Hide Original' : 'Show Original',
            icon: Icon(_showOriginal ? Icons.visibility : Icons.visibility_off),
          ),
          IconButton(
            onPressed: () => setState(() {
              _captureWithoutEditor = !_captureWithoutEditor;
            }),
            tooltip: _captureWithoutEditor ? 'Use Editor' : 'Without Editor',
            icon: Icon(
                _captureWithoutEditor ? Icons.edit_off_rounded : Icons.edit),
          ),
        ],
      ),
      body: Stack(
        children: [
          Center(
            child: _imageBytes != null
                ? Image.memory(_imageBytes!)
                : const Text('No image selected.'),
          ),
          if (_showOriginal && _originalBytes != null)
            Center(child: Image.memory(_originalBytes!))
        ],
      ),
      floatingActionButton: _imageBytes == null
          ? null
          : FloatingActionButton.extended(
              onPressed: _openEditor,
              label: const Text('+ 10 Edits'),
            ),
      bottomNavigationBar: Padding(
        padding: const EdgeInsets.all(16.0),
        child: ElevatedButton(
            onPressed: _pickImage,
            child: const Text('Pick Image from Gallery')),
      ),
    );
  }
}

[isenbj]

I've had to move on to other things right now as my timebox for this ran out. We are definitely in need of this fix though. My current workaround is to just warn the users that it is a known bug and will be addressed in a future update.

Hoping to get some time allotted to look into this more, but hoping you can help with a resolution.

[barchinvictor]

Hi! I also see this problem. Please prioritize this problem

[timekone]

Hi! Thanks for this awesome library! ❤️

I can reliably reproduce this issue when running on web platform as well.
Package version 4.2.7, Flutter version 3.22.2
Unfortunately, this is a pretty serious bug.

[hm21]

@timekone

Thank you for your kind words. I'm glad you like my package.

Resolving this issue is now a top priority for me. However, I'm currently very busy with other work projects, so I don't have enough time to fix it. If you need a quicker fix, you are welcome to create a pull request. Alternatively, you can review a simplified version of my code that I posted here. In that example, you can remove the code part where it uses the image editor and also adjust the pixel ratio. This should give you a minimal working example of the image editor in just a few lines of code. If you identify the issue, feel free to let me know, and I can update the editor myself, in the case it doesn't require too much time.

[Unicorn254]

Hi! Thanks for this awesome library! ❤️

I can reliably reproduce this issue when running on mobile platform as well.
Package version 6.0.0, Flutter version 3.24.3

[saif-ellafi]

I came to share the pains here, to me the pain is mostly due to image layers and stickers that I resize smaller than the background image. Upon saving, those layers suffer catastrophic quality loss, so when resizing back to make them bigger, they're essentially destroyed.

For now the only approach that helped, with performance impact, is artificially expand the customPixelRatio so it sticks to the base image (plus a multiplier), and use png and 0 compression.

I might need to implement something to load layers using their original source image rather their compressed implementation.