Refurbishment #118
Refurbishment #118
Conversation
* Rename tools Change the `Performs` in the name of the two tools - `Performs projective transformation with/without labels` - `Performs projective transformation` to `Perform`, which is more consistent with the name of the other tools (imperative form). * Update projective_transformation.xml * Update projective_transformation_points.xml * Add CONVENTIONS.md * wip * wip * Update CONVENTIONS.md * Update CONVENTIONS.md * Update CONVENTIONS.md * Update CONVENTIONS.md * Update CONVENTIONS.md * Update CONVENTIONS.md * Remove help section * Update CONVENTIONS.md * Update CONVENTIONS.md * Update CONVENTIONS.md * Update * Update test data * Add content assertions * Update pr.yaml * Update * Update pr.yaml * Add `pip install pillow` * Merge CONVENTIONS.md into CONTRIBUTING.md * Temporarilly fix CI * Update `2d_feature_extraction` tool * Update `2d_auto_threshold` tool * Fix `2d_feature_extraction` tests * Fix test data * Fix tool * Start refactoring `2d_simple_filter` tool * Fix `2d_simple_filter` tool * Add more tests * Fix tests * Update * Fix * Update CONTRIBUTING.md * Fix bugs * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Fix linting issues * Update CONTRIBUTING.md * Refactor tests using macros * Update tests * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update pr.yaml * Update `anisotropic_diffusion` tool * Fix spelling (was BE, using AE is more consistent) * Update `2d_histogram_equalization` tool * Update `colorize_labels` tool * Update `bf2raw` tool * Add `macros/creators.xml` * Update `rfove` tool * Fix `bioformats2raw` and `voronoi_tessellation` tools * Fix labels * Fix `rfove` tool * Update `overlay_images` tool * Add RGB test for `overlay_images` * Update `overlay_images` help * Add image verification support to CI (BMCV#117) * Update pr.yaml * Update * Update pr.yaml * Add `pip install pillow` * Temporarilly fix CI * Update pr.yaml * Update ci.yaml * Add test tool from `/test/functional/tools/image_diff.xml` * Update pr.yaml * Add .shed.yml * Add missing test files * Switch CI to galaxy fork `kostrykin/galaxy` branch `galaxy-image-analysis` This is a temporary change to enable the features from: - galaxyproject/galaxy#17556 - galaxyproject/galaxy#17581 * Remove `test` tool * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update test data * Add RGB test for `overlay_images` * Update `anisotropic_diffusion` to medpy 0.4.0 * Update PULL_REQUEST_TEMPLATE.md * Update `projective_transformation_points` tool * Update PULL_REQUEST_TEMPLATE.md * Update `bioformats2raw` * Update `projective_transformation` tool * Update `bfconvert` tool (tests failing) * Update CONTRIBUTING.md * Fix `bfconvert` tests * Fix test data size * Update `projective_transformation_points` tool * Remove spurious files * Update requirements using `@TOOL_VERSION@` macro * Update `segmetrics` tool * Remove spurious sym links * Update `superdsm` tool * Update `scale_image` tool (not finished yet) * Fix tests * Fix `scale_image` tool * Refactor * Fix linting issues * Update tests and help of `2d_simple_filter` tool * Update `scale_image` help * Update CI to use latest version of `galaxy` fork * Fix tests for float TIFF in `scale_image` tool * Add tifffile to CI workflows * Fix tests of `bfconvert` tool * Update `2d_filter_segmentation_by_features` tool * Update `image_math` tool * Update `orientationpy` tool * Update `morphological_operations` tool * Update `split_labelmap` tool * Add float TIFF test for `2d_simple_filter` * Update `slice_image` tool * Add missing macros * Update help of `2d_filter_segmentation_by_features` tool * Update `concat_images` tool * Fix linting issues * Fix file sizes * Fix missing `profile`
| @@ -0,0 +1,23 @@ | |||
| <macros> | |||
There was a problem hiding this comment.
Please note that the entire file is shipped to all tools.
| @@ -0,0 +1,93 @@ | |||
| <macros> | |||
There was a problem hiding this comment.
Doesn't this make it very unreadable for the actual tool?
And also here those macro files are shipped to all tools that import them, even if they do not use most of the stuff ... and it gets parsed and loaded into Galaxy I think.
There was a problem hiding this comment.
I'm torn, I like how compact it is, but I'm worried those files are growing and growing and single tools don't use much of it.
There was a problem hiding this comment.
Thanks for the feedback!
On the other hand, if the macros were split into more fine-grained files, this would result in more I/O operations for loading the tools. Do you think that this is tolerable and parsing a few superfluous macros is worse?
My concern is due to: I remember from my C++ days, that, for compilers, I/O operations are the reason while compile times are long. By concatenating all files into a single file (aka unitybuild), compile times could be reduced by >90% (the improvement was somewhat less when using SSD instead of HDD but still huge).
There was a problem hiding this comment.
I understand, but I guess this is not whats happening here. Every tool will ship the macro, so the number of files is the same in both cases. I one scenario its just a lot small files, and in your case its potentially a lot of big files with a lot of information that is not used for individual tools.
Its an academic discussion, please keep it like it is, I just wanted to point out this disadvantage.
There was a problem hiding this comment.
This is 0 Bytes and other as well, is that ok?
There was a problem hiding this comment.
The file is in fact 85.4 KB in size, not 0 :)
https://github.com/BMCV/galaxy-image-analysis/blob/6904d1db7414422d8e41f1d5b3f5ee45ef9e9e79/tools/2d_auto_threshold/test-data/out3.tif
There was a problem hiding this comment.
True, sorry fo the noise.
|
Very cool! |
Now that image-based test verification is supported, it is time to refurbish our tools and eliminate bugs or inconsistencies which before went undetected.
The following tools are updated in this PR:
2d_auto_threshold2d_feature_extraction2d_filter_segmentation_by_features2d_histogram_equalization2d_simple_filteranisotropic_diffusionbfconvertbioformats2rawcolorize_labelsconcat_channelsimage_mathmorphological_operationsorientationpyoverlay_imagesprojective_transformationprojective_transformation_pointsrfovescale_imagesegmetricsslice_imagesplit_labelmapsuperdsmvoronoi_tessellationTo avoid boilerplate code and provide meaningful defaults for different types of images, macros for testing are defined. These are documented in CONTRIBUTING.md.
In addition, on the occasion of the updates, the help sections of many tools have been revised, and the
creatorsof the tool wrappers have been added.The following bugs and inconsistencies were fixed:
scale_imageand2d_simple_filterwill now preserve the brightness and the range of values of images (for2d_simple_filterthis only concerns mean filters like Gaussian and median). This will be handy for such applications.scale_imagenow supports TIFF files.concat_imagesnow supports either preserving the range of values or the image brightness.Last but not least, the CI is updated to also support the TIFF file features added in galaxyproject/galaxy#17797.