Skip to content

Commit

Permalink
sphinx/links-and-formatting fixup More touch-ups to docs, also remove…
Browse files Browse the repository at this point in the history
… most funky capitalisation there
  • Loading branch information
pgunn committed Feb 21, 2024
1 parent 0297127 commit 729b984
Show file tree
Hide file tree
Showing 10 changed files with 58 additions and 61 deletions.
8 changes: 4 additions & 4 deletions docs/source/CaImAn_Tips.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
CaImAn Tips
===========
General Tips
============

Motion Correction tips
----------------------
Expand Down Expand Up @@ -33,7 +33,7 @@ Motion Correction tips
of chunks (``params.motion.splits_els`` and ``params.motion.split_rig``)
with the length of your recording (e.g. ``int((number of total frames)/200)``).

CaImAn Online Processing tips
Caiman Online Processing tips
-----------------------------

- Important parameters for online processing are the CNN threshold
Expand Down Expand Up @@ -71,7 +71,7 @@ CaImAn Online Processing tips
- When using the CNN for screening candidate components, the usage of a
GPU can lead to significant computational gains.

CaImAn Batch processing tips
Caiman Batch processing tips
----------------------------

- In order to optimize memory consumption and parallelize computing, it
Expand Down
7 changes: 3 additions & 4 deletions docs/source/CaImAn_features_and_references.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Caiman Features
---------------

CaImAn includes a variety of scalable methods for the analysis of
Caiman includes a variety of scalable methods for the analysis of
calcium (and voltage) imaging data:

|
Expand Down Expand Up @@ -44,7 +44,7 @@ calcium (and voltage) imaging data:

- Uses a Mask R-CNN to identify neurons in the FOV
- Extracts spiking activity using adaptive template matching.
- Fully integrated with CaImAn, inherits all its capabilities.
- Fully integrated with Caiman, inherits all its capabilities.
|
- **Behavioral Analysis** [Behavior]_

Expand All @@ -59,8 +59,7 @@ calcium (and voltage) imaging data:

- Noise parameters estimation under the Poisson-Gaussian noise model
- Fast algorithm that scales to large datasets
- A basic demo can be found at
``CaImAn/demos/notebooks/demo_VST.ipynb``
- A basic demo can be found in the default demos as demo_VST.ipynb



Expand Down
8 changes: 4 additions & 4 deletions docs/source/GUI.rst
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
Graphical interfaces
====================

The CaImAn GUI
---------------
The Caiman GUI
--------------

.. image:: ../img/GUI_img.png
:width: 600px
:align: center

CaImAn comes with an experimental visual interface. To see an example on
Caiman comes with an experimental visual interface. To see an example on
how use it, first load and run either \* demo_OnACID_mesoscope.py \*
demo_caiman_basic.py \* demo_pipeline.py

Expand Down Expand Up @@ -37,7 +37,7 @@ More features will be added in the future.
External interfaces
-------------------

A couple of very nice visualization tools that directly call CaImAn have been developed recently
A couple of very nice visualization tools that directly call Caiman have been developed recently
by external researchers. For more information please check the following projects:

- `mesmerize-core <https://github.com/nel-lab/mesmerize-core>`_ - also simplifies grid search for parameter tuning
Expand Down
35 changes: 16 additions & 19 deletions docs/source/Getting_Started.rst
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
Getting Started with CaImAn
Getting Started
===========================

Demos
-----

- Notebooks: The notebooks provide a simple and friendly way to get
into CaImAn and understand its main characteristics. They are located
in the ``demos/notebooks``. To launch one of the jupyter notebooks:
into Caiman and understand its main characteristics. They are located
in the ``demos/notebooks``. To launch one of the jupyter notebooks, activate your conda caiman environment, enter the caiman_data directory, and then:

.. code:: bash
source activate CaImAn
jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10
and select the notebook from within Jupyter’s browser. The argument
Expand All @@ -21,9 +20,6 @@ Demos
trying demo_pipeline.py first as it contains most of the tasks
required by calcium imaging. For behavior use demo_behavior.py

- If you want to directly launch the python files, your python console
still must be in the CaImAn directory.

Basic Structure
---------------

Expand All @@ -36,13 +32,13 @@ the following objects:
easily and is passed into all the algorithms.
- ``MotionCorrect``: An object for motion correction which can be used
for both rigid and piece-wise rigid motion correction.
- ``cnmf``: An object for running the CaImAn batch algorithm either in
- ``cnmf``: An object for running the Caiman batch algorithm either in
patches or not, suitable for both two-photon (CNMF) and one-photon
(CNMF-E) data.
- ``online_cnmf``: An object for running the CaImAn online (OnACID)
- ``online_cnmf``: An object for running the Caiman online (OnACID)
algorithm on two-photon data with or without motion correction.
- ``estimates``: A single object that stores the results of the
algorithms (CaImAn batch, CaImAn online) in a unified way that also
algorithms (batch, online) in a unified way that also
contains plotting methods.

To see examples of how these methods are used, please consult the demos.
Expand All @@ -51,7 +47,7 @@ To see examples of how these methods are used, please consult the demos.
Parameters
-----------

CaImAn gives you access to a lot of parameters and lets you adapt the analysis to your data. Parameters are stored in
Caiman gives you access to a lot of parameters and lets you adapt the analysis to your data. Parameters are stored in
the ``params`` object in a set of dictionaries, sorted by the part of the analysis they are used in:

- ``data``: General params describing the dataset like dimensions, decay time, filename and framerate
Expand Down Expand Up @@ -123,7 +119,7 @@ As mentioned above, the results of the analysis are stored within the
Result variables for 2p batch analysis
--------------------------------------

The results of CaImAn are saved in an ``estimates`` object. This is
The results of Caiman are saved in an ``estimates`` object. This is
stored inside the cnmf object, i.e. it can be accessed using
``cnmf.estimates``. The variables of interest are:

Expand Down Expand Up @@ -305,19 +301,20 @@ The cluster is set up with Caiman's ``setup_cluster()`` function, which takes in
n_processes=None,
ignore_preexisting=False)

The ``backend`` parameter determines the type of cluster used. The default value, ``'multiprocessing'``, uses the
multiprocessing package, but ``ipyparallel`` is also available. More information on these choices can be
found `here <https://github.com/flatironinstitute/CaImAn/blob/master/docs/CLUSTER.md>`_. You can set the number of
processes (cpu cores) to use with the ``n_processes`` parameter: the default value ``None`` will lead to the function
selecting one *less* than the total number of logical cores available.
The **backend** parameter determines the type of cluster used. The default value, **'multiprocessing'**, uses the
multiprocessing package, but **ipyparallel** is also available. You can set the number of
processes (cpu cores) to use with the **n_processes** parameter: the default value **None** will lead to the function
selecting one *less* than the total number of logical cores available.

More information on these choices can be found :doc:`in the cluster doc <cluster>`.

The parameter ``ignore_preexisting``, which defaults to ``False``, is a failsafe used to avoid overwhelming your resources.
If you try to start another cluster when Caiman already has one running, you will get an error. However, sometimes
on more powerful machines you may want to spin up multiple Caiman environments. In that case,
set ``ignore_preexisting`` to ``True``.

The output variable ``cluster`` is the multicore processing object that will be used in subsequent processing steps. It will
be passed as a parameter in subsequent stages and is the fulcrum for parallelization. The
be passed as a parameter in subsequent stages and sets policy for parallelization. The
other output that can be useful to check is ``n_processes``, as it will tell you how many CPU cores you have set up
in your cluster.

Expand Down Expand Up @@ -347,4 +344,4 @@ will be read/written by column or by row, respectively. This is important becaus
faster on C-order arrays vs F-order arrays. For motion correction, which needs to access contiguous sequences of
frames (often in the middle of the movie), it is much more efficient to read and write in F order. On the other
hand, when it comes to CNMF, you need to access individual pixels across the entire movie, so Caiman saves the
motion-corrected movie in C-order before running CNMF.
motion-corrected movie in C-order before running CNMF.
2 changes: 1 addition & 1 deletion docs/source/Handling_Movies.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Handling movies
================

This page gives a basic intro on the movie object for loading, manipulating,
displaying and saving movies within CaImAn.
displaying and saving movies.

Loading movies
--------------
Expand Down
21 changes: 11 additions & 10 deletions docs/source/Installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,7 @@ Section 1B. Development-mode install
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Dev mode install allows you to modify the source files of Caiman and makes it easier
to contribute to the project, fix bugs etc. The general motivation for setting up
an editable development environment is described in more detail in our `contributors page <https://github.com/flatironinstitute/CaImAn/blob/main/CONTRIBUTING.md>`_.
to contribute to the project, fix bugs etc.

If you install in dev mode you will likely need to set some environment variables manually (it is
done automatically when you do the conda install): this is discussed in Section 4C.
Expand Down Expand Up @@ -107,7 +106,7 @@ At the conda prompt:

.. code:: bash
git clone https://github.com/your-username/CaImAn
git clone git@github.com:flatironinstitute/CaImAn.git
cd CaImAn
mamba env create -f environment.yml -n caiman
conda activate caiman
Expand Down Expand Up @@ -140,7 +139,7 @@ Dev Mode Install on MacOS and Linux

.. code:: bash
git clone https://github.com/your-username/CaImAn
git clone git@github.com:flatironinstitute/CaImAn.git
cd CaImAn/
mamba env create -f environment.yml -n caiman
source activate caiman
Expand Down Expand Up @@ -235,7 +234,7 @@ below to update the code. However, if the environment file has changed
since your last update this may lead to you not the latest version. None of this applies
to the conda-forge route (for which instructions are given above).

From the conda environment you used to install CaImAn:
From the conda environment you used to install Caiman:

1. ``pip uninstall caiman``

Expand All @@ -257,7 +256,7 @@ From the conda environment you used to install CaImAn:
variables have changed.

- The same applies if you want to modify some internal function of
CaImAn. If you used the ``pip install -e .`` option then you can
Caiman. If you used the ``pip install -e .`` option then you can
directly modify it (that's why it's the editable developer mode). If you
used the ``pip install .`` option then you will need to
``pip uninstall caiman`` followed by ``pip install .`` for your
Expand All @@ -278,7 +277,7 @@ Section 3C: Upgrade the demos with caimanmanager
<summary>Upgrade the demos</summary>

When you upgrade Caiman, sometimes the underlying APIs change. When this happens and it impacts a demo (or otherwise
requires changes to files in ``caiman_data``), we update the demo and data. This means that upgrading CaImAn works
requires changes to files in ``caiman_data``), we update the demo and data. This means that upgrading Caiman works
best if you also replace the ``caiman_data`` directory with a new version.

To check if the demos or datafiles have changed since your last install, you can run ``caimanmanager check``. If they have not,
Expand Down Expand Up @@ -372,6 +371,8 @@ following the instructions

Section 4D: Other topics
~~~~~~~~~~~~~~~~~~~~~~~~~
See also
- Our clustering doc
- Caiman and GPUs
See also:

* :doc:`Our clustering doc <cluster>`
* :doc:`Caiman and GPUs <readme-gpu>`

4 changes: 2 additions & 2 deletions docs/source/Overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ Overview
:width: 300px
:align: right

CaImAn is a Python toolbox for large scale **Ca**\ lcium **Im**\ aging data **An**\ alysis and behavioral analysis.
Caiman is a Python toolbox for large scale **Ca**\ lcium **Im**\ aging data **An**\ alysis and behavioral analysis.

CaImAn implements a set of essential methods required in the analysis pipeline of large scale calcium imaging data. Fast and scalable algorithms are implemented for motion correction, source extraction, spike deconvolution, and component registration across multiple days. It is suitable for both two-photon and one-photon fluorescence microscopy data, and can be run in both batch and online modes. CaImAn also contains some routines for the analysis of behavior from video cameras. A list of features as well as relevant references can be found `here
Caiman implements a set of essential methods required in the analysis pipeline of large scale calcium imaging data. Fast and scalable algorithms are implemented for motion correction, source extraction, spike deconvolution, and component registration across multiple days. It is suitable for both two-photon and one-photon fluorescence microscopy data, and can be run in both batch and online modes. It also contains some routines for the analysis of behavior from video cameras. A list of features as well as relevant references can be found `here
<CaImAn_features_and_references.html>`_.

Companion paper
Expand Down
6 changes: 3 additions & 3 deletions docs/source/cluster.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
On Clustering
=============

Some parts of the CaImAn library (and demos) are best done by farming work out over multiple CPUs (or in some cases, multiple separate systems in a cluster). Python provides (at least) two popular frameworks for this, multiprocessing and ipyparallel. Both of them have several modes and parameters, each with their own advantages and disadvantages. Their behaviour differs on different operating systems (e.g. Linux versus OSX) and different Python environments (script versus Spyder versus Jupyter). CaImAn provides some support for some modes of both frameworks, to make it easier for people to find a setup that works on their platform.
Some parts of the Caiman library (and demos) are best done by farming work out over multiple CPUs (or in some cases, multiple separate systems in a cluster). Python provides (at least) two popular frameworks for this, multiprocessing and ipyparallel. Both of them have several modes and parameters, each with their own advantages and disadvantages. Their behaviour differs on different operating systems (e.g. Linux versus OSX) and different Python environments (script versus Spyder versus Jupyter). It provides some support for some modes of both frameworks, to make it easier for people to find a setup that works on their platform.

This document covers some of the reasons you might choose one framework over another, how you can pick a clustering option with the codebase, and some of the issues we have seen with each.

Expand Down Expand Up @@ -30,6 +30,6 @@ for the cluster. The parameter can take the following documented values:
Issues
------
* On OSX, some math libraries (blas, lapack) by default use a hardware-accelerated system framework that interacts badly with threads and might not work if run from a forked process. This causes some worker processes to hang forever if more than one of them tries to do some computations at the same time. You may be able to prevent this with "export VECLIB_MAXIMUM_THREADS=1" before running your code, or telling Conda to install a blas/lapack that is not built against Veclib.
* On Linux and OSX, some multithreaded builds of math libraries do their own multithreading and this can break if CaImAn forks to make new processes. This can result in hangs and/or core dumps. We have in our instructions other environment variables you should set to tell them not to multithread.
* On Linux and OSX, some multithreaded builds of math libraries do their own multithreading and this can break if Caiman forks to make new processes. This can result in hangs and/or core dumps. We have in our instructions other environment variables you should set to tell them not to multithread.
* OpenCV is not easy to use safely with multithreaded or multi-process software. We take steps to disable multithreading in OpenCV, but some builds of it may not allow this. If you run into these issues let us know; we may ask you to switch the OpenCV version you use (Conda makes this easy).
* Some builds of OpenCV use OpenMP, which by default opens a thread for each of your CPUs for background work. This multiplies out to a potentially large number if you have already started many processes for CaImAn work. If this is both true and a problem for your environment, you should limit OpenMP to a single thread by doing the following before running CaImAn code: "export OMP_NUM_THREADS=1", or alternatively switch to a different build of OpenCV.
* Some builds of OpenCV use OpenMP, which by default opens a thread for each of your CPUs for background work. This multiplies out to a potentially large number if you have already started many processes for Caiman work. If this is both true and a problem for your environment, you should limit OpenMP to a single thread by doing the following before running Caiman code: "export OMP_NUM_THREADS=1", or alternatively switch to a different build of OpenCV.
Loading

0 comments on commit 729b984

Please sign in to comment.