Update nightly #136
Open
Update nightly #136
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
anthonyprinaldi
commented
Jul 3, 2024
anthonyprinaldi
commented
- Merge all recent changes from dev (ACE related) into nightly release
- This is in preparation for testing the new Python version in the nightly branch
This fix makes sure that the right atlas is selected by the registration script, instead of the workflow. This is needed for when using half brain registration.
Add registration overwrite ability to single workflow.
Clear reg_final/ and clar_allen_reg/ when user choses to rerun registration
Add parser flag to skip segmentation when the segmentation has already been run.
Seg skipping was only added to comparison workflow and not single workflow.
In Tiff to Nifti allow the user to specify a value of percentile to correct extreme artifacts in the image. Default no correction.
Let ACE use the percentil correction in Tiff to Nifti conversion to crop extreme intensities.
This was the problem with the registration function. We also no longer need to perform the percentile correction.
If labels are specified, use the user specified argument. Otherwise, come up with the labels based on other arguments.
sw_batch_size is hard coded to 4. batch_size is now taken from user input.
This allows us to specify the number of MC iterations instead of hard coding the value.
Need to move results to CPU before doing numpy stackin. Otherwise results in an error
New parser argument to select number of GPU to be used.
Deal with outputs when batch size is greater than 1.
Ensure that the variable of the outside loop doesn't conflict with the list comprehension
Need to keep original values of val_outputs for future loop iterations.
Statements originally included for debugging no longer needed.
Add flag for forcing rerun conversion. By default conversion will be skipped if it already completed and the right downsample arg is used.
The stacked patches are only beign zero padded in the height and width dimension right now. We need to pad the depth dimension so that all patches loaded to the model are the same size.
The warp script does not have ability to specify custom allen labels (in the case of half-brain registratin). Add the flag that overwrite the default allen atlas labels.
The output of MC ace models were being saved with additional uncertaintiy information. To handle this, we need to filter the output seg folder for specific files. These files re named different for regular vs MC ace models.
Sometimes a None string is passed through the CLI that we must check for, otherwise the script will try and load the path None. We check if the label path is not empty and not None
Ace heatmap used to be for whole brain only. This commit adds logic to load in half brain atlas labels if half brain is passed to the ACE workflow. This also adds a flag for heatmap that masks out voxels that our outside of the brain, as determined by the atlas.
The model output by default will determine if a voxel is a neuron with 0.5 threshold. We now add an argument so we can change this threshold to be whatever we want.
Previously, creating output storage arrays based on the assumed patch size of 512^3. We now take the patch size from the input so that it can work with any patch size.
Function will plot the p-values from ACE clusterwise analysis and overlay it onto a brain template from the atlas.
ACE flow will now plot a p-value overlaid on an atalas after clusterwise analysis is complete.
Add validate clusters and instance segmentation output to ace workflow
Second Round of ACE Fixes.
Add ACE Finetuning Script
…all.sh Some users have to run Docker with "sudo" as they don't have a "docker" group. This will cause problems with the install script as it will try to create a user "root" inside of the container. "root" being the default Docker user, this will fail. Users are therefore warned that the installation script can only be used if Docker can be used without "sudo" on their system. The next step here would be to add a check in the installation script that throws an error when the detected uid is "root".
install.sh needs to be updated with new functionality based on user feedback. Before implementing the user feedback though it is useful to first refactor the script. This will make it easier to implement the new features.
The interactive prompt will guide the user through the most important arguments that have to be provided to the installation script. The prompt is an abbreviated version of the cli flags. The reasoning behind that is that users who need to use the prompt instead of providing the flags are probably not advanced enought to use the extra flags in the first place.
One problem that users ran into was duplicated image/container names. Sometimes they would install MIRACL but then use the install.sh script to install MIRACL again without first removing the previous Docker image/container. This would lead to build conflicts. This could be solved if users used the -i and -c flags to generate custom names. However, it can't be assumed that user do that so now instead of a hard coded default name, a random default name is created every time the script is run. This should prevent conflicts when running the install.sh script again. The downside of this is that a new version of MIRACL will be created every time the installation script is being run...
User feedback showed that the prompts in the code examples should be excluded when using the copybutton. Changed the settings for sphinx-copybutton to exclude the '$' prompt in our code examples. Also autoformatted the config file.
If the user uses the interactive prompt, there will now be an option to set the location for the data on the host system that will then automatically be mounted at '/data' within the container. Also fixed numerous smaller issues as well as inconsistencies caused by the new interactive prompt.
These additions make the installation section a lot more verbose. Additions and changes were made based on user feedback. The installation instruction also contain a section on the new, interactive installation method now.
Python's typer package is automatically being installed as part of PyTorch. However, the Typer version installed was > 0.12.0 which changed its dependency structure, removing the need for typer[all]. After version 0.12.0, the typer package uses typer-slim[standard] which includes the default dependencies, instead of typer[all]. This introduced breaking changes to MIRACL and the installed version of PyTorch. Pinning the version to 0.11.1 solved the issue. Also moved up torchio to more clearly cluster it with other PyTorch packages.
[fix] pin typer to 0.11.1 to fix MIRACL breaking changes introduced by ACE
[docs/feat] improve docs based on user feedback
[feat/fix] significant re-write of installation script
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.