Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function to mask tissue regions #104

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

LLehner
Copy link
Collaborator

@LLehner LLehner commented Nov 11, 2024

Once ready, closes #50

Adds a function to mask an image. Given a user-provided shape, either crop the image such that only pixels within the mask remain or if Invert=False mask the image (only keep pixels outside the shape).

example calls for image masking with user provided shape:

mask_image(sdata, "test", "mask", invert=True)

masked_image_1

mask_image(sdata, "test", "mask", invert=False)

masked_image_2

Alternatively one can use a threshold + watershed approach to automatically segment and mask the image.
E.g. given the following test image:

circular

mask_image(sdata, "test_2", None, invert=True, automatic_masking=True, threshold=50)

yields

auto_mask_50

with threshold=80:

auto_mask_80

For automatic region detection the threshold is image-specific.

Right now the code is still a draft, since i wasn't sure how to properly embed it into your current codebase, so the main questions are:

  1. what does the input look like (e.g. instead of spatialdata it will be a project object, but how does it look like when calling the function and how to access the sdata object?).
  2. Are the images always grayscale?
  3. Should the cropped image be completely resized or will setting masked pixels to 0 suffice?
  4. is the mask_filtering module the correct place for this function?
  5. Should the automatic approach be more robust and use more novel object detection algorithms or allow for users to plug their own methods in?

@LLehner LLehner marked this pull request as draft November 11, 2024 23:24
@sophiamaedler
Copy link
Collaborator

Looks like a great start already! In what form does the user need to provide a shape at the moment? can this be done interactively in Napari?

  1. what does the input look like (e.g. instead of spatialdata it will be a project object, but how does it look like when calling the function and how to access the sdata object?).

I would implement this as a method to the project object so that we can e.g. call:
project.mask_input_image(mask, ....)

Under the hood this would then access the spatialdata object that is associated with the project file. You can get it via:
sdata = self.filehandler.get_sdata()

So I guess an implementation along the lines of:

def mask_input_image(mask: ..., invert: bool = False):
    sdata = self.filehandler.get_sdata()
    masked = mask_image(sdata, "test", "mask", invert=True)
    self.filehandler. _write_image_sdata(masked, 
        "input_image",
        channel_names=...,
        scale_factors=....,
        chunks=(1, 1000, 1000),
        overwrite=True)

Currently the _write_image_sdata is still a function belonging to project but it needs to be relocated (and improved) to the filehandler. (the filehandler was introduced more recently when I realized that we otherwise encounter issues with multithreading some operations but the migration has not been completed in full yet).

  1. Are the images always grayscale?

So far we have never worked with RGB images. @namsaraeva is working on this for the first time now.
But at the moment I think its fine to implement only for images of the form CXY.

  1. Should the cropped image be completely resized or will setting masked pixels to 0 suffice?

Should be completely resized as this will improve run time of all downstream steps. We have some checks where code is not executed if an area only contains 0 pixels, but these checks only go into effect if NO non-zero values are contained.

  1. is the mask_filtering module the correct place for this function?

mask_filtering module here refers to operations to filter generated segmentation masks (e.g. to only keep masks with a min cell size). I think the best place for it would be under pipeline/_utils/helper.py.

  1. Should the automatic approach be more robust and use more novel object detection algorithms or allow for users to plug their own methods in?

I think it would be nice to have the possibility for users to plug in their own methods, but I think this is a functionality which will only be used by a few more advanced users. So the default as you implemented it now with a thresholding should be sufficient.

Polygon containing the detected regions.
"""
if image.shape[0] == 1:
image = image[0]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to improve thresholding here a little we could calculate this on a downsampled version of the image which has also been blurred slightly. This should make it a little easier to detect macro structures (like e.g. tissue) somewhat smoothly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tissue Region masking in scPortrait
2 participants