Suppress non-fatal errors when loading images (#119)

* Add failing test case to reproduce an issue * Add `imread` wrapper to make the test pass * Switch to `giatools` support package * Update CONTRIBUTING.md * Update `2d_feature_extraction` tool * Update `2d_filter_segmentation_by_features` tool * Update `2d_histogram_equalization` tool * Update `2d_simple_filter` tool * Update `anisotropic_diffusion` tool * Update `colorize_labels` tool * Update `concat_channels` tool * Update `image_math` tool * Update `morphological_operations` tool * Update `orientationpy` tool * Update `overlay_images` tool * Update `projective_transformation` tool * Update `scale_image` tool * Update `slice_image` tool * Update `split_labelmap` tool * Update `voronoi_tessellation` tool * Fix `slice_image` and `split_labelmap` tools (profile was missing) * Update `superdsm` tool
BMCV · Apr 24, 2024 · c86a1b9 · c86a1b9
1 parent 4631aaa
commit c86a1b9
Show file tree

Hide file tree

Showing 40 changed files with 110 additions and 71 deletions.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -3,7 +3,7 @@
 
 **FOR THE CONTRIBUTOR — Please fill out if applicable**
 
-Please make sure you have read the [CONTRIBUTING.md](https://github.com/BMCV/galaxy-image-analysis/blob/master/CONTRIBUTING.md) document (last updated: 2024/03/18).
+Please make sure you have read the [CONTRIBUTING.md](https://github.com/BMCV/galaxy-image-analysis/blob/master/CONTRIBUTING.md) document (last updated: 2024/04/23).
 
 * [ ] License permits unrestricted use (educational + commercial).
 

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -8,7 +8,7 @@ This document is the attempt to collect some rough rules for tools to follow in
 * Make sure you have git [installed](https://help.github.com/articles/set-up-git)
 * Fork the repository on [GitHub](https://github.com/BMCV/galaxy-image-analysis/fork)
 * Make the desired modifications - consider using a [feature branch](https://github.com/Kunena/Kunena-Forum/wiki/Create-a-new-branch-with-git-and-manage-branches).
-* Try to stick to the [Conventions for Tools in the Image Community](https://github.com/elixir-europe/biohackathon-projects-2023/blob/main/16/paper/paper.md#conventions) and the [IUC standards](http://galaxy-iuc-standards.readthedocs.org/en/latest/) whenever you can
+* Try to stick to the [Conventions for Tools in the Image Community](https://doi.org/10.37044/osf.io/w8dsz) and the [IUC standards](http://galaxy-iuc-standards.readthedocs.org/en/latest/) whenever you can
 * Make sure you have added the necessary tests for your changes and they pass.
 * Open a [pull request](https://help.github.com/articles/using-pull-requests) with these changes.
 
@@ -22,18 +22,18 @@ This document is the attempt to collect some rough rules for tools to follow in
 
 ## File types
 
-If a tool wrapper only supports single-channel 2-D images and uses a Python script, the structure of the input should be verified right after loading the image:
-
+In tool wrappers which use a Python script, image loading should be performed by using the `giatools` package (see https://github.com/BMCV/galaxy-image-analysis/pull/119).
+If such wrappers only support single-channel 2-D images, the structure of the input should be verified right after loading the image:
 ```python
-im = skimage.io.imread(args.input)
+im = giatools.io.imread(args.input)
 im = np.squeeze(im)  # remove axes with length 1
 assert im.ndim == 2
 ```
 
 Tools with **label map inputs** should accept PNG and TIFF files. Tools with **label map outputs** should produce either `uint16` single-channel PNG or `uint16` single-channel TIFF. Using `uint8` instead of `uint16` is also acceptable, if there definetely are no more than 256 different labels. Using `uint8` should be preferred for binary images.
 
 > [!NOTE]  
-> It is a common misconception that PNG files must be RGB or RGBA, and that only `uint8` pixel values are supported. For example, the `cv2` module (OpenCV) can be used to create single-channel PNG files, or PNG files with `uint16` pixel values. Such files can then be read by `skimage.io.imread` without issues (however, `skimage.io.imwrite` seems not to be able to write such PNG files).
+> It is a common misconception that PNG files must be RGB or RGBA, and that only `uint8` pixel values are supported. For example, the `cv2` module (OpenCV) can be used to create single-channel PNG files, or PNG files with `uint16` pixel values. Such files can then be read by `giatools.io.imread` or `skimage.io.imread` without issues (however, `skimage.io.imwrite` seems not to be able to write such PNG files).
 
 Tools with **intensity image inputs** should accept PNG and TIFF files. Tools with **intensity image outputs** can be any data type and either PNG or TIFF. Image outputs meant for visualization (e.g., segmentation overlays, charts) should be PNG.
 

diff --git a/tools/2d_auto_threshold/auto_threshold.py b/tools/2d_auto_threshold/auto_threshold.py
@@ -7,12 +7,13 @@
 
 import argparse
 
+import giatools.io
 import numpy as np
 import skimage.filters
-import skimage.io
 import skimage.util
 import tifffile
 
+
 th_methods = {
     'manual': lambda thres, **kwargs: thres,
 
@@ -28,7 +29,7 @@
 
 
 def do_thresholding(in_fn, out_fn, th_method, block_size, offset, threshold, invert_output=False):
-    img = skimage.io.imread(in_fn)
+    img = giatools.io.imread(in_fn)
     img = np.squeeze(img)
     assert img.ndim == 2
 

diff --git a/tools/2d_auto_threshold/auto_threshold.xml b/tools/2d_auto_threshold/auto_threshold.xml
@@ -4,7 +4,7 @@
         <import>creators.xml</import>
         <import>tests.xml</import>
         <token name="@TOOL_VERSION@">0.18.1</token>
-        <token name="@VERSION_SUFFIX@">2</token>
+        <token name="@VERSION_SUFFIX@">3</token>
         <xml name="inputs/offset">
             <param name="offset" type="float" value="0" label="Offset" help="Offset to be added to the automatically determined threshold value. Positive values will increase the threshold (and thus reduce the amount of values above the threshold)." />
         </xml>
@@ -22,6 +22,7 @@
     <requirements> 
         <requirement type="package" version="@TOOL_VERSION@">scikit-image</requirement>
         <requirement type="package" version="2020.10.1">tifffile</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements>
     <command detect_errors="aggressive">
     <![CDATA[
@@ -124,6 +125,13 @@
             <param name="invert_output" value="True"/>
             <expand macro="tests/binary_image_diff" name="output" value="out4.tif" ftype="tiff"/>
         </test>
+        <!-- Tests for irregular files -->
+        <test>
+            <param name="input" value="sample2.tif"/>
+            <param name="method_id" value="otsu"/>
+            <param name="invert_output" value="False"/>
+            <expand macro="tests/binary_image_diff" name="output" value="out5.tif" ftype="tiff"/>
+        </test>
         <!-- Tests for multi-channel images -->
         <test expect_failure="true">
             <param name="input" value="rgb.png"/>

diff --git a/tools/2d_auto_threshold/test-data/out5.tif b/tools/2d_auto_threshold/test-data/out5.tif
diff --git a/tools/2d_auto_threshold/test-data/sample2.tif b/tools/2d_auto_threshold/test-data/sample2.tif
diff --git a/tools/2d_feature_extraction/2d_feature_extraction.py b/tools/2d_feature_extraction/2d_feature_extraction.py
@@ -1,9 +1,9 @@
 import argparse
 
+import giatools.io
 import numpy as np
 import pandas as pd
 import skimage.feature
-import skimage.io
 import skimage.measure
 import skimage.morphology
 import skimage.segmentation
@@ -57,9 +57,9 @@
 
     raw_image = None
     if args.raw_file is not None:
-        raw_image = skimage.io.imread(args.raw_file.name)
+        raw_image = giatools.io.imread(args.raw_file.name)
 
-    raw_label_image = skimage.io.imread(label_file)
+    raw_label_image = giatools.io.imread(label_file)
 
     df = pd.DataFrame()
     if label_file_binary:
@@ -124,4 +124,4 @@
         df['convexity'] = area / (perimeter * perimeter)
 
     del df['it']
-    df.to_csv(out_file, sep='\t', line_terminator='\n', index=False)
+    df.to_csv(out_file, sep='\t', lineterminator='\n', index=False)
diff --git a/tools/2d_feature_extraction/2d_feature_extraction.xml b/tools/2d_feature_extraction/2d_feature_extraction.xml
@@ -2,7 +2,7 @@
     <description>with scikit-image</description>
     <macros>
         <import>creators.xml</import>
-        <token name="@TOOL_VERSION@">0.14.2</token>
+        <token name="@TOOL_VERSION@">0.18.1</token>
         <token name="@VERSION_SUFFIX@">0</token>
     </macros>
     <creator>
@@ -17,9 +17,10 @@
     </xrefs>
     <requirements> 
         <requirement type="package" version="@TOOL_VERSION@">scikit-image</requirement>
-        <requirement type="package" version="0.23.4">pandas</requirement>
-        <requirement type="package" version="1.15.4">numpy</requirement>
-        <requirement type="package" version="0.15.1">tifffile</requirement>
+        <requirement type="package" version="2.2.2">pandas</requirement>
+        <requirement type="package" version="1.23.3">numpy</requirement>
+        <requirement type="package" version="2020.10.1">tifffile</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements>
     <command detect_errors="aggressive">
     <![CDATA[

diff --git a/tools/2d_filter_segmentation_by_features/2d_filter_segmentation_by_features.py b/tools/2d_filter_segmentation_by_features/2d_filter_segmentation_by_features.py
@@ -1,6 +1,7 @@
 import argparse
 import sys
 
+import giatools.io
 import pandas as pd
 import skimage.io
 import skimage.util
@@ -14,14 +15,14 @@
     parser.add_argument('rule_file', type=argparse.FileType('r'), default=sys.stdin, help='file with rules per feature (cols: ,f1,2, rows: feature_name, min, max)')
     args = parser.parse_args()
 
-    img_in = skimage.io.imread(args.input_file.name)
+    img_in = giatools.io.imread(args.input_file.name)
     features = pd.read_csv(args.feature_file, delimiter="\t")
     rules = pd.read_csv(args.rule_file, delimiter="\t")
 
     cols = [a for a in rules.columns if 'Unnamed' not in a]
     for a_c in cols:
-        a_min = rules[rules.ix[:, 0] == 'min'][a_c]
-        a_max = rules[rules.ix[:, 0] == 'max'][a_c]
+        a_min = rules[rules.iloc[:, 0] == 'min'][a_c]
+        a_max = rules[rules.iloc[:, 0] == 'max'][a_c]
         for a_l in features.label:
             a_val = float(features[features['label'] == a_l][a_c])
             if a_val < float(a_min) or a_val > float(a_max):

diff --git a/tools/2d_filter_segmentation_by_features/2d_filter_segmentation_by_features.xml b/tools/2d_filter_segmentation_by_features/2d_filter_segmentation_by_features.xml
@@ -1,4 +1,4 @@
-<tool id="ip_2d_filter_segmentation_by_features" name="Filter label map by rules" version="0.0.1-3" profile="20.05">
+<tool id="ip_2d_filter_segmentation_by_features" name="Filter label map by rules" version="0.0.1-4" profile="20.05">
     <description></description>
     <macros>
         <import>creators.xml</import>
@@ -14,10 +14,10 @@
         <xref type="bio.tools">galaxy_image_analysis</xref>
     </xrefs>
     <requirements>
-        <requirement type="package" version="0.14.2">scikit-image</requirement> 
-        <requirement type="package" version="5.3.0">pillow</requirement>
-        <requirement type="package" version="0.23.4">pandas</requirement>
-        <requirement type="package" version="0.15.1">tifffile</requirement>
+        <requirement type="package" version="0.18.1">scikit-image</requirement> 
+        <requirement type="package" version="2.2.2">pandas</requirement>
+        <requirement type="package" version="2020.10.1">tifffile</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements>
     <command detect_errors="aggressive"><![CDATA[
 

diff --git a/tools/2d_histogram_equalization/histogram_equalization.py b/tools/2d_histogram_equalization/histogram_equalization.py
@@ -1,6 +1,7 @@
 import argparse
 import sys
 
+import giatools.io
 import skimage.exposure
 import skimage.io
 import skimage.util
@@ -17,7 +18,7 @@
     parser.add_argument('h_type', choices=hOptions.keys(), help='histogram equalization method')
     args = parser.parse_args()
 
-    img_in = skimage.io.imread(args.input_file.name)
+    img_in = giatools.io.imread(args.input_file.name)
     res = hOptions[args.h_type](img_in)
     res = skimage.util.img_as_uint(res)
     skimage.io.imsave(args.out_file.name, res, plugin="tifffile")
diff --git a/tools/2d_histogram_equalization/histogram_equalization.xml b/tools/2d_histogram_equalization/histogram_equalization.xml
@@ -3,7 +3,7 @@
     <macros>
         <import>creators.xml</import>
         <import>tests.xml</import>
-        <token name="@TOOL_VERSION@">0.14.2</token>
+        <token name="@TOOL_VERSION@">0.18.1</token>
         <token name="@VERSION_SUFFIX@">0</token>
     </macros>
     <creator>
@@ -18,9 +18,9 @@
     </xrefs>
     <requirements> 
         <requirement type="package" version="@TOOL_VERSION@">scikit-image</requirement>
-        <requirement type="package" version="1.15.4">numpy</requirement>
-        <requirement type="package" version="5.3.0">pillow</requirement>
-        <requirement type="package" version="0.15.1">tifffile</requirement>
+        <requirement type="package" version="1.23.3">numpy</requirement>
+        <requirement type="package" version="2020.10.1">tifffile</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements>
     <command detect_errors="aggressive">
     <![CDATA[

diff --git a/tools/2d_simple_filter/filter.xml b/tools/2d_simple_filter/filter.xml
@@ -4,7 +4,7 @@
         <import>creators.xml</import>
         <import>tests.xml</import>
         <token name="@TOOL_VERSION@">1.12.0</token>
-        <token name="@VERSION_SUFFIX@">0</token>
+        <token name="@VERSION_SUFFIX@">1</token>
     </macros>
     <creator>
         <expand macro="creators/bmcv" />
@@ -20,6 +20,7 @@
         <requirement type="package" version="1.26.4">numpy</requirement>
         <requirement type="package" version="0.22.0">scikit-image</requirement>
         <requirement type="package" version="2024.2.12">tifffile</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements>
     <command detect_errors="aggressive"><![CDATA[
 

diff --git a/tools/2d_simple_filter/filter_image.py b/tools/2d_simple_filter/filter_image.py
@@ -1,5 +1,6 @@
 import argparse
 
+import giatools.io
 import scipy.ndimage as ndi
 import skimage.io
 import skimage.util
@@ -23,6 +24,6 @@
     parser.add_argument('size', type=float, help='Size of the filter (e.g., radius, sigma)')
     args = parser.parse_args()
 
-    im = skimage.io.imread(args.input.name)
+    im = giatools.io.imread(args.input.name)
     res = filters[args.filter](im, args.size)
     skimage.io.imsave(args.output.name, res, plugin='tifffile')
diff --git a/tools/anisotropic_diffusion/anisotropic_diffusion.py b/tools/anisotropic_diffusion/anisotropic_diffusion.py
@@ -2,6 +2,7 @@
 import sys
 import warnings
 
+import giatools.io
 import skimage.io
 import skimage.util
 from medpy.filter.smoothing import anisotropic_diffusion
@@ -18,7 +19,7 @@
 with warnings.catch_warnings():
     warnings.simplefilter("ignore")  # to ignore FutureWarning as well
 
-    img_in = skimage.io.imread(args.input_file.name, plugin='tifffile')
+    img_in = giatools.io.imread(args.input_file.name, plugin='tifffile')
     res = anisotropic_diffusion(img_in, niter=args.niter, kappa=args.kappa, gamma=args.gamma, option=args.eqoption)
     res[res < -1] = -1
     res[res > +1] = +1

diff --git a/tools/anisotropic_diffusion/anisotropic_diffusion.xml b/tools/anisotropic_diffusion/anisotropic_diffusion.xml
@@ -4,7 +4,7 @@
         <import>creators.xml</import>
         <import>tests.xml</import>
         <token name="@TOOL_VERSION@">0.4.0</token>
-        <token name="@VERSION_SUFFIX@">0</token>
+        <token name="@VERSION_SUFFIX@">1</token>
     </macros>
     <creator>
         <expand macro="creators/bmcv" />
@@ -16,6 +16,7 @@
         <requirement type="package" version="@TOOL_VERSION@">medpy</requirement>
         <requirement type="package" version="1.20">numpy</requirement>
         <requirement type="package" version="0.18.1">scikit-image</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements> 
     <command detect_errors="aggressive">
         <![CDATA[

diff --git a/tools/colorize_labels/colorize_labels.py b/tools/colorize_labels/colorize_labels.py
@@ -1,5 +1,6 @@
 import argparse
 
+import giatools.io
 import matplotlib.colors as mpl
 import networkx as nx
 import numpy as np
@@ -55,7 +56,7 @@ def build_label_adjacency_graph(im, radius, bg_label):
     args = parser.parse_args()
 
     # Load image and normalize
-    im = skimage.io.imread(args.input)
+    im = giatools.io.imread(args.input)
     im = np.squeeze(im)
     assert im.ndim == 2
 

diff --git a/tools/colorize_labels/colorize_labels.xml b/tools/colorize_labels/colorize_labels.xml
@@ -4,7 +4,7 @@
         <import>creators.xml</import>
         <import>tests.xml</import>
         <token name="@TOOL_VERSION@">3.2.1</token>
-        <token name="@VERSION_SUFFIX@">1</token>
+        <token name="@VERSION_SUFFIX@">2</token>
     </macros>
     <creator>
         <expand macro="creators/bmcv" />
@@ -16,6 +16,7 @@
         <requirement type="package" version="@TOOL_VERSION@">networkx</requirement>
         <requirement type="package" version="1.22">numpy</requirement>
         <requirement type="package" version="0.18.1">scikit-image</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements>
     <command><![CDATA[
     

diff --git a/tools/concat_channels/concat_channels.py b/tools/concat_channels/concat_channels.py
@@ -1,5 +1,6 @@
 import argparse
 
+import giatools.io
 import numpy as np
 import skimage.io
 import skimage.util
@@ -9,7 +10,7 @@ def concat_channels(input_image_paths, output_image_path, axis, preserve_values)
     images = []
     for image_path in input_image_paths:
 
-        raw_image = skimage.io.imread(image_path)
+        raw_image = giatools.io.imread(image_path)
         if len(raw_image.shape) == 2:
             if axis == 0:
                 raw_image = [raw_image]

diff --git a/tools/concat_channels/concat_channels.xml b/tools/concat_channels/concat_channels.xml
@@ -1,4 +1,4 @@
-<tool id="ip_concat_channels" name="Concatenate images or channels" version="0.3-0" profile="20.05">
+<tool id="ip_concat_channels" name="Concatenate images or channels" version="0.3-1" profile="20.05">
     <description></description>
     <macros>
         <import>creators.xml</import>
@@ -17,6 +17,7 @@
         <requirement type="package" version="0.18.3">scikit-image</requirement>
         <requirement type="package" version="1.24.1">numpy</requirement>
         <requirement type="package" version="2021.7.2">tifffile</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements>
     <command detect_errors="aggressive"><![CDATA[
 

diff --git a/tools/image_math/image_math.py b/tools/image_math/image_math.py
@@ -2,6 +2,7 @@
 import ast
 import operator
 
+import giatools.io
 import numpy as np
 import skimage.io
 
@@ -77,7 +78,7 @@ def eval_expression(expr, inputs):
     im_shape = None
     for input in args.input:
         name, filepath = input.split(':')
-        im = skimage.io.imread(filepath)
+        im = giatools.io.imread(filepath)
         assert name not in inputs, 'Input name "{name}" is ambiguous.'
         inputs[name] = im
         if im_shape is None:

diff --git a/tools/image_math/image_math.xml b/tools/image_math/image_math.xml
@@ -4,7 +4,7 @@
         <import>creators.xml</import>
         <import>tests.xml</import>
         <token name="@TOOL_VERSION@">1.26.4</token>
-        <token name="@VERSION_SUFFIX@">1</token>
+        <token name="@VERSION_SUFFIX@">2</token>
     </macros>
     <creator>
         <expand macro="creators/bmcv"/>
@@ -15,6 +15,7 @@
     <requirements>
         <requirement type="package" version="@TOOL_VERSION@">numpy</requirement>
         <requirement type="package" version="0.22.0">scikit-image</requirement>
+        <requirement type="package" version="0.1">giatools</requirement>
     </requirements>
     <command><![CDATA[