Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete conversion pipeline: code refactoring #10

Merged
merged 16 commits into from
Oct 24, 2024

Conversation

alessandratrapani
Copy link
Collaborator

@alessandratrapani alessandratrapani commented Oct 23, 2024

  • Add support for all data streams -> Zaki 2024 Conversion (Cai lab) #1
  • Refactor code -> add all interfaces to interfaces folder
  • Correct spelling errors
  • Add auxiliary function to add motion correction and global ids

NB: how to identify global ids needs to be discussed with the author

I will add the conversion script in a follow up PR

@alessandratrapani alessandratrapani self-assigned this Oct 23, 2024
@alessandratrapani alessandratrapani marked this pull request as ready for review October 23, 2024 11:02
Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR was a tad too big : )

src/cai_lab_to_nwb/zaki_2024/utils/cell_registration.py Outdated Show resolved Hide resolved
nwbfile: NWBFile,
motion_correction_series: np.ndarray,
one_photon_series_name: str,
convert_to_dtype: DtypeType = None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I can add the motion correction in the Converter once the OnePhotoSeries has been added

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My question was not very clear, I am talking about convert_to_dtype.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry! No we don't need it, I copied and paste it accidentally

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you reference the original implementation if you copied it from somewhere else?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, is it our thing from one of our repos then. Thanks.

src/cai_lab_to_nwb/zaki_2024/utils/motion_correction.py Outdated Show resolved Hide resolved
assert (
num_frames == motion_correction_series.shape[0]
), f"The number of frames for motion correction ({motion_correction_series.shape[0]}) does not match the number of frames ({num_frames}) from the {one_photon_series_name} imaging data."
xy_translation = TimeSeries(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be a OnePhtonSeries as well?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, for the motion correction we have a specific neurodatatype that store the xt shift as a timeseries and link to the original imaging data

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So why are we using TimeSeries and not the specific neurodatatype? I am confused.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I am confused too apparently...
I wanted to add the MotionCorrection container, but then I decided to store just the xy_shif as a TimeSeries to keep it light.

@@ -14,6 +16,10 @@ def __init__(self, file_path: FilePath, verbose: bool = False):
self.verbose = verbose
super().__init__(file_path=file_path)

def get_timestamps_reference_time(self):
Copy link
Collaborator

@h-mayorquin h-mayorquin Oct 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_session_start_time to be in line with the other methods? What do you think?

I don't remember this is "means" object already a datetime object? maybe add a return type or docstring

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to differentiate from the session start time because this would not be the session start time for this conversion. It's just an auxiliary function to get the time reference of the first timestamp of these traces. I will use this to slice the traces once we understand with the author how they want to align this data.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the case for all the other interfaces. We are never sure if they will be the session_start_time, it is only when you use that interface as stand-alone that it should work that way.

OK, but that makes sense. We will probably not generalize this code outside of this conversion.

Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

feel free to merge.

@h-mayorquin h-mayorquin merged commit d856b1c into main Oct 24, 2024
@h-mayorquin h-mayorquin deleted the refactor_conversion_pipeline branch October 24, 2024 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants