-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Lauren M Ostrowski
committed
Jul 15, 2024
1 parent
44b79e3
commit d46b6c8
Showing
3 changed files
with
535 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # Chronic ephys pipeline documentation | ||
|
|
||
| Pipeline built in the Gentner Lab that takes in chronic single-unit electrophysiology, audio data, and other forms of data streams, syncs streams together, and handles automated and hand curated spike sorting and song analysis. The following notebooks are provided: | ||
|
|
||
| ## 1-preprocess_acoustics | ||
|
|
||
| This notebook handles the initial preprocessing of neural data from SpikeGLX and OpenEphys as well as other simultaneous data streams (audio or pressure). | ||
|
|
||
| Environment to use: **songprep**. This notebook and *2-curate_acoustics* rely on Zeke Arneodo's pipeline <u>ceciestunepipe</u>. [Ceciestunepipe](https://github.com/laurenostrowski/ceciestunepipe) handles loading and syncing neural and audio data, but requires certain early versions of published packages that have since been updated. The environment **songprep** is compatible with ceciestunepipe, and is therefore used for preprocessing steps, but should not be used for forward analyses. | ||
|
|
||
| #### Errors you might encounter | ||
| - **Number of edges in the syn ttl events of pattern and target don't match:** sy.sync_all will throw an error if streams are different lengths. For example, if the neuropixel comes unplugged during a recording session, NIDQ and WAV streams will be of the same length, different than the length of LFP and AP streams (see bird: z_c5o30_23, sess: 2023-08-12 for an example). This is because NIDQ contains a microphone channel that will continue recording even if the neuropixel data is disrupted. | ||
| - **Events array for lf_0 had skipped heartbeats:** sy.sync_all will throw an error if it detects that heartbeats were skipped in any of the data streams. We have configured our system to send a 0.5 Hz square wave pulse to align the machine clocks for all data streams. If it detects periods between square wave pulses (i.e., heartbeats) are of different lengths, it might mean that the signal was momentarily lost. | ||
|
|
||
| ## 2-curate_acoustics | ||
|
|
||
| This notebook provides an interface for you to manually curate automatically detected bouts of song. You can manually inspect putative bouts and trim the bout start and stop indices (precise alignment to the bout will improve the performance of the syllable curation algorithm later on). | ||
|
|
||
| Environment to use: **songprep**. | ||
|
|
||
| ## 3-sort_spikes | ||
|
|
||
| This notebook runs an automatic spike sorting algorithm on the neural data, defaulting to the spike sorting algorithm [Kilosort 3](https://kilosort.readthedocs.io/en/latest/). *Kilosort 4 was released on March 3rd, 2024 and will be added to the notebook as the default sorting algorithm when integrated into SpikeInterface*. | ||
|
|
||
| Environment to use: **spikeprep**. | ||
|
|
||
| ## 4-curate_spikes | ||
|
|
||
| This notebook allows you to manually curate the outputs of an automatic spike sorting algorithm using the web-based viewer provided by [SpikeInterface](https://spikeinterface.readthedocs.io/en/latest/). | ||
|
|
||
| Environment to use: **spikeprep**. | ||
|
|
||
| #### Errors you might encounter | ||
| - **ValueError: Out of range float values are not JSON compliant:** you might get an error message resembling this when running *si.plot_quality_metrics* if one or more quality metrics could not be computed in automatic spike sorting. Skip the metrics that could not be computed (using the spike sort log as a guide) using the flag skip_metrics=['metrics', 'to', 'skip'] | ||
|
|
||
| ## 5-cluster_acoustics | ||
|
|
||
| This *optional* notebook... | ||
|
|
||
| ## 6-convert_to_NWB | ||
|
|
||
| This notebook converts the outputs of processing pipeline to the [Neuroscience without Borders (NWB) Data Standard](https://www.nwb.org/nwb-neurophysiology/) formatting. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| name: songproc | ||
| channels: | ||
| - defaults | ||
| dependencies: | ||
| - _libgcc_mutex=0.1=main | ||
| - _openmp_mutex=5.1=1_gnu | ||
| - anyio=3.5.0=py38h06a4308_0 | ||
| - argon2-cffi=21.3.0=pyhd3eb1b0_0 | ||
| - argon2-cffi-bindings=21.2.0=py38h7f8727e_0 | ||
| - asttokens=2.0.5=pyhd3eb1b0_0 | ||
| - attrs=22.1.0=py38h06a4308_0 | ||
| - backcall=0.2.0=pyhd3eb1b0_0 | ||
| - beautifulsoup4=4.11.1=py38h06a4308_0 | ||
| - bleach=4.1.0=pyhd3eb1b0_0 | ||
| - ca-certificates=2023.01.10=h06a4308_0 | ||
| - certifi=2022.12.7=py38h06a4308_0 | ||
| - cffi=1.15.1=py38h5eee18b_3 | ||
| - debugpy=1.5.1=py38h295c915_0 | ||
| - decorator=5.1.1=pyhd3eb1b0_0 | ||
| - defusedxml=0.7.1=pyhd3eb1b0_0 | ||
| - entrypoints=0.4=py38h06a4308_0 | ||
| - executing=0.8.3=pyhd3eb1b0_0 | ||
| - flit-core=3.6.0=pyhd3eb1b0_0 | ||
| - icu=58.2=he6710b0_3 | ||
| - idna=3.4=py38h06a4308_0 | ||
| - importlib_resources=5.2.0=pyhd3eb1b0_1 | ||
| - ipykernel=6.19.2=py38hb070fc8_0 | ||
| - ipython=8.8.0=py38h06a4308_0 | ||
| - ipython_genutils=0.2.0=pyhd3eb1b0_1 | ||
| - jedi=0.18.1=py38h06a4308_1 | ||
| - jinja2=3.1.2=py38h06a4308_0 | ||
| - jsonschema=4.16.0=py38h06a4308_0 | ||
| - jupyter_client=7.4.8=py38h06a4308_0 | ||
| - jupyter_core=5.1.1=py38h06a4308_0 | ||
| - jupyter_server=1.23.4=py38h06a4308_0 | ||
| - jupyterlab_pygments=0.1.2=py_0 | ||
| - ld_impl_linux-64=2.38=h1181459_1 | ||
| - libffi=3.4.2=h6a678d5_6 | ||
| - libgcc-ng=11.2.0=h1234567_1 | ||
| - libgomp=11.2.0=h1234567_1 | ||
| - libsodium=1.0.18=h7b6447c_0 | ||
| - libstdcxx-ng=11.2.0=h1234567_1 | ||
| - libxml2=2.9.14=h74e7548_0 | ||
| - libxslt=1.1.35=h4e12654_0 | ||
| - lxml=4.9.1=py38h1edc446_0 | ||
| - markupsafe=2.1.1=py38h7f8727e_0 | ||
| - matplotlib-inline=0.1.6=py38h06a4308_0 | ||
| - mistune=0.8.4=py38h7b6447c_1000 | ||
| - nbclassic=0.4.8=py38h06a4308_0 | ||
| - nbclient=0.5.13=py38h06a4308_0 | ||
| - nbconvert=6.5.4=py38h06a4308_0 | ||
| - nbformat=5.7.0=py38h06a4308_0 | ||
| - ncurses=6.4=h6a678d5_0 | ||
| - nest-asyncio=1.5.6=py38h06a4308_0 | ||
| - notebook=6.5.2=py38h06a4308_0 | ||
| - notebook-shim=0.2.2=py38h06a4308_0 | ||
| - openssl=1.1.1s=h7f8727e_0 | ||
| - packaging=22.0=py38h06a4308_0 | ||
| - pandocfilters=1.5.0=pyhd3eb1b0_0 | ||
| - parso=0.8.3=pyhd3eb1b0_0 | ||
| - pexpect=4.8.0=pyhd3eb1b0_3 | ||
| - pickleshare=0.7.5=pyhd3eb1b0_1003 | ||
| - pip=22.3.1=py38h06a4308_0 | ||
| - pkgutil-resolve-name=1.3.10=py38h06a4308_0 | ||
| - platformdirs=2.5.2=py38h06a4308_0 | ||
| - prometheus_client=0.14.1=py38h06a4308_0 | ||
| - prompt-toolkit=3.0.36=py38h06a4308_0 | ||
| - psutil=5.9.0=py38h5eee18b_0 | ||
| - ptyprocess=0.7.0=pyhd3eb1b0_2 | ||
| - pure_eval=0.2.2=pyhd3eb1b0_0 | ||
| - pycparser=2.21=pyhd3eb1b0_0 | ||
| - pygments=2.11.2=pyhd3eb1b0_0 | ||
| - pyrsistent=0.18.0=py38heee7806_0 | ||
| - python=3.8.16=h7a1cb2a_2 | ||
| - python-dateutil=2.8.2=pyhd3eb1b0_0 | ||
| - python-fastjsonschema=2.16.2=py38h06a4308_0 | ||
| - pyzmq=23.2.0=py38h6a678d5_0 | ||
| - readline=8.2=h5eee18b_0 | ||
| - send2trash=1.8.0=pyhd3eb1b0_1 | ||
| - setuptools=65.6.3=py38h06a4308_0 | ||
| - six=1.16.0=pyhd3eb1b0_1 | ||
| - sniffio=1.2.0=py38h06a4308_1 | ||
| - soupsieve=2.3.2.post1=py38h06a4308_0 | ||
| - sqlite=3.40.1=h5082296_0 | ||
| - stack_data=0.2.0=pyhd3eb1b0_0 | ||
| - terminado=0.17.1=py38h06a4308_0 | ||
| - tinycss2=1.2.1=py38h06a4308_0 | ||
| - tk=8.6.12=h1ccaba5_0 | ||
| - tornado=6.2=py38h5eee18b_0 | ||
| - traitlets=5.7.1=py38h06a4308_0 | ||
| - typing-extensions=4.4.0=py38h06a4308_0 | ||
| - typing_extensions=4.4.0=py38h06a4308_0 | ||
| - wcwidth=0.2.5=pyhd3eb1b0_0 | ||
| - webencodings=0.5.1=py38_1 | ||
| - websocket-client=0.58.0=py38h06a4308_4 | ||
| - wheel=0.37.1=pyhd3eb1b0_0 | ||
| - xz=5.2.10=h5eee18b_1 | ||
| - zeromq=4.3.4=h2531618_0 | ||
| - zipp=3.11.0=py38h06a4308_0 | ||
| - zlib=1.2.13=h5eee18b_0 | ||
| - pip: | ||
| - appdirs==1.4.4 | ||
| - audioread==3.0.0 | ||
| - charset-normalizer==3.0.1 | ||
| - comm==0.1.4 | ||
| - contourpy==1.0.7 | ||
| - cycler==0.11.0 | ||
| - elephant==0.12.0 | ||
| - fonttools==4.38.0 | ||
| - h5py==3.8.0 | ||
| - importlib-metadata==6.0.0 | ||
| - ipympl==0.9.3 | ||
| - ipywidgets==8.1.0 | ||
| - joblib==1.1.1 | ||
| - jupyterlab-widgets==3.0.8 | ||
| - kiwisolver==1.4.4 | ||
| - librosa==0.9.2 | ||
| - llvmlite==0.39.1 | ||
| - matplotlib==3.6.3 | ||
| - more-itertools==9.0.0 | ||
| - neo==0.11.1 | ||
| - networkx==3.0 | ||
| - noisereduce==2.0.1 | ||
| - numba==0.56.4 | ||
| - numpy==1.23.5 | ||
| - pandas==1.5.3 | ||
| - parse==1.19.0 | ||
| - pathlib2==2.3.7.post1 | ||
| - peakutils==1.3.4 | ||
| - pickle5==0.0.11 | ||
| - pillow==9.4.0 | ||
| - plotly==5.16.1 | ||
| - pooch==1.6.0 | ||
| - probeinterface==0.2.16 | ||
| - pyparsing==3.0.9 | ||
| - pytz==2022.7.1 | ||
| - quantities==0.14.1 | ||
| - requests==2.28.2 | ||
| - resampy==0.4.2 | ||
| - scikit-learn==1.2.1 | ||
| - scipy==1.10.0 | ||
| - seaborn==0.12.2 | ||
| - soundfile==0.11.0 | ||
| - spikeextractors==0.9.3 | ||
| - spikeinterface==0.96.1 | ||
| - tenacity==8.2.3 | ||
| - threadpoolctl==3.1.0 | ||
| - tqdm==4.64.1 | ||
| - umap==0.1.1 | ||
| - urllib3==1.26.14 | ||
| - viziphant==0.3.0 | ||
| - widgetsnbextension==4.0.8 | ||
| prefix: /mnt/cube/lo/envs/songproc |
Oops, something went wrong.