Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation read through #219

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
4c780cb
Minor changes to installation instructions
Oct 15, 2024
db7cfaa
Attempt to fix cppp coverage issue
Oct 15, 2024
42a7e8f
Attempting to fix code coverage error
Oct 15, 2024
77df555
Attempting to fix code coverage error with additional line
Oct 15, 2024
ce34fd1
revert testing test
Oct 15, 2024
ea8c2e5
Further attempt to fix coverage error
Oct 15, 2024
6195c88
Moving lcov stuff to other branch
Oct 15, 2024
b4d8b2d
Merge branch 'main' into issue/216/documentation-read-through
Oct 16, 2024
48b2277
Remove unnecesary installation line
Oct 16, 2024
1852844
Merge branch 'main' into issue/216/documentation-read-through
Oct 21, 2024
26b7161
Improved installation instructions to include notebook requirements
Oct 21, 2024
47e423a
Add known issue surrounding pypi installation from source
Oct 21, 2024
3140de1
Added known issue regarding AVX2
Oct 22, 2024
abd5a72
changes to getting started from Olivier's reading
Oct 23, 2024
e1f22b8
Heavy changes in the detailed documentation to adapt from the overlea…
Nov 8, 2024
e78906b
Merge remote-tracking branch 'origin/improved_detaileddoc' into issue…
Nov 9, 2024
f507ebe
Updates to intermediate notebook following Oliver comments in issue
Nov 11, 2024
93c0fc3
Implemented Olivier changes to example full run notebook
Nov 11, 2024
9c2275e
Updates to legacy install page following Olivier notes
Nov 11, 2024
612c9a6
Minor changes to getting started
Nov 11, 2024
e55c6a9
Olivier edits to keywords
Nov 11, 2024
7dffc22
Minor changes from discussion
Nov 12, 2024
c7ab75e
Update the skim figure from Olivier
Nov 12, 2024
1ecfd44
Various changes and restructuring
Nov 12, 2024
3e35c7f
Merge branch 'main' into issue/216/documentation-read-through
Nov 12, 2024
803fac8
Other minor changes based on discussion
Nov 12, 2024
7b69b07
Some changes to notebooks following discussion
Nov 12, 2024
37cdbc1
More detail on input table in the typical use case notebook
Nov 13, 2024
e4489b0
Minor changes to typical use case
Nov 13, 2024
64eeacb
Readthrough of typical use case notebook
Nov 13, 2024
8ac337b
Lots of changes to notebooks and docs
Nov 13, 2024
c438c68
Fix bug on star ext law
Nov 13, 2024
05724c5
Update warning regarding environment variables
Nov 13, 2024
3b0eb44
Merging changes from main
Nov 13, 2024
8bdff0d
Minor change as merge had conflicts
Nov 13, 2024
0153b41
Consistent naming
Nov 13, 2024
e55f43e
Remove duplicated content in the Python usage section
Nov 14, 2024
928a3ec
Changes in the detailed documentation after Mara comments
Nov 14, 2024
0b88beb
Merge remote-tracking branch 'origin/documentation-read-through' into…
Nov 14, 2024
2a7ee2f
Very minor change following Olivier's branch merge
Nov 14, 2024
d41e403
Remove warning regarding version to make this the official repo
Nov 15, 2024
8552b4c
Trying to fix documentation build issue it seems that GitH
Nov 15, 2024
28265b2
Updating pooch tests to deal with new downloader
Nov 15, 2024
299f143
Further tests on retrieval with user specified
Nov 15, 2024
7c13667
Got original tests working with new downloader specified
Nov 15, 2024
7339ca2
In the python usage page, point to the README rather than a specific …
Nov 21, 2024
416f717
Minor changes in the text of index and getting_started
Nov 21, 2024
7a048b6
slight improvements in advanced_install
Nov 21, 2024
bac19f8
Include comments from Mara and other changes from Olivier
Nov 22, 2024
7a7075b
Minor typo fix
Nov 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@ LePHARE is a code for estimating galaxy redshifts and physical parameters using

The code documentation, including how to install it and to get started, can be found [here](https://lephare.readthedocs.io/).

**IMPORTANT!** The current repository is under development. If you wish to use LePHARE for science work, please download it from the [currently stable repository](https://gitlab.lam.fr/Galaxies/LEPHARE/).

# Requests and help

If you need help with the code, or if you have feature requests, please use the github issue system to let us know.
Expand Down
4 changes: 4 additions & 0 deletions docs/_static/custom.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.wy-table-responsive table td,
.wy-table-responsive table th {
white-space: normal;
}
159 changes: 159 additions & 0 deletions docs/advanced_install.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
Advanced Installation with full clones of repositories
======================================================
LePHARE is split across two repositories one for the
`code <https://github.com/lephare-photoz/lephare>`_ and one for the
auxiliary `data <https://github.com/lephare-photoz/lephare-data>`_.
Advanced users or developers may want to edit the files in
both. In this section we explain a developer install and clone of both
git repositories in order to allow modification of the code and auxiliary data.


Auxiliary Data and the Environment Variables
********************************************

LePHARE depends on auxiliary data sets such as spectral energy distributions,
filter transmission curves, and attenuation curves. In order to keep the pip
installation light these are now stored in a distinct repository called
`lephare-data <https://github.com/lephare-photoz/lephare-data>`_.

LePHARE uses environment variables to locate the external data and work files. These are set by default to your cache. Both can be set if preferred. The two environment variables are the following:

* `LEPHAREDIR` is the location of the auxiliary data.
* `LEPHAREWORK` is the location of the intermediate files produced during a LePHARE run.



.. note::
If you want to download the external data to a specific location you must set the
environment variable `LEPHAREDIR` to its location. This must be done prior to
importing lephare in a python session. If not LePHARE will use the default cache
location on your system.

Some users may prefer to simply clone the entire auxiliary data directory:

.. code-block:: bash

git clone https://github.com/lephare-photoz/lephare-data
# Set the LEPHAREDIR to this data location
export LEPHAREDIR=$PWD/lephare-data


We have also built some automatic machinery for downloading the
required files for a given config `para` file. The automatic download
functionality can also be used to download all external data. In the
following snippet we show how you might set the `LEPHAREDIR` to a new
location and download all the auxiliary data there:

.. code-block:: python

import os
os.environ['LEPHAREDIR']='/path/to/my/preferred/data_directory/'
os.environ['LEPHAREWORK']='/path/to/my/preferred/working_directory/'
# You must import lephare after setting the variables
import lephare as lp
# If you do not set a config input to the following function in gets everything.
# Data will be put in $LEPHAREDIR
lp.data_retrieval.get_auxiliary_data(clone=False)
# Setting clone=True would use a git clone which may be faster but will only run
# on an empty directory.


Developer Installation
**********************
The developer install is required for editing the code but can also be useful
on systems that do not have PyPI binaries and for systems that are not well tested.
Before installing any dependencies or writing code, it's a great idea to create
a virtual environment. LINCC-Frameworks engineers primarily use conda to manage
virtual environments. If you have conda installed locally, you can run the following
to create and activate a new environment. We then recommend installing in
editable mode with the `-e` option so that any changes are immediately propagated.

.. tabs::

.. tab:: bash

.. code-block:: bash

conda create -n <env_name> python=3.12
conda activate <env_name>
git clone https://github.com/lephare-photoz/lephare.git
cd lephare
git submodule update --init --recursive
conda install -c conda-forge cxx-compiler
pip install -e .'[dev]'

.. tab:: OSX

.. code-block:: bash

conda create -n <env_name> python=3.12
conda activate <env_name>
brew install llvm libomp
git clone https://github.com/lephare-photoz/lephare.git
cd lephare
git submodule update --init --recursive
conda install -c conda-forge cxx-compiler
pip install -e .'[dev]'


At this stage running the tests is a good way to check everything is working:

.. code-block:: bash

python -m pytest tests

Once you have created a new environment, you can install precommit and pandoc
which will help you to run precommit checks and create the documentation locally:

.. code-block:: bash

pre-commit install
conda install pandoc

Developers can also build the documentation in the following way:

.. code-block:: bash

cd docs/
pip install -r requirements.txt #install sphinx dependencies
make html

The doc entry will then be located at `../_readthedocs/html/index.html`. The
documentation includes a rendering of the notebooks, which thus need to be
executed. You can bypass this stage by replacing `make html`` above by
`make no-notebooks`. Executing `make` will list further options.


If you wish to incorporate your changes to the main branch, please make a fork of
the repository and then create a pull request.

If you are having problems with installations, there is a list of known issues `here <known_issues.rst>`_.
If you can’t find a solution, feel free to `create an issue in the lephare repository
<https://github.com/lephare-photoz/lephare/issues>`_.

Some developers who are familiar with the original version of the code may
want to have all the external data present in the same repository as the code
or some other preferred location. They could set the `LEPHAREDIR` to the code
location and then use the automatic downloading functionality to put all
the auxiliary data there as it was in the previous versions.


.. note::
The single quotes around `'[dev]'` may not be required for your operating system.

`pre-commit install` will initialize pre-commit for this local repository,
so that a set of tests will be run prior to completing a local commit. For more
information, see the Python Project Template documentation on `pre-commit
<https://lincc-ppt.readthedocs.io/en/latest/practices/precommit.html>`_.

Installing `pandoc` allows you to verify that automatic rendering of Jupyter
notebooks into documentation for ReadTheDocs works as expected. For more information,
see the Python Project Template documentation on `Sphinx and Python Notebooks
<https://lincc-ppt.readthedocs.io/en/latest/practices/sphinx.html#python-notebooks>`_.

The environment variables `LEPHAREDIR` and `LEPHAREWORK` are set on import
in Python. Care must be taken not to reset after importing.

It remains possible to build the C++ code using either make or cmake directly.
This is not recommended and will likely require OS specific changes. It may be
useful on unusual systems where we do not support compilation.
113 changes: 113 additions & 0 deletions docs/commandline_usage.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
Advanced Usage via Command Line Interface
=========================================

The four principle command line tools are immediately available after installing
with pip. They can also be called by building the c++ executables directly but
that is generally not advised. The four basic operations are summarised below.

Running with the command line
-----------------------------

LePHARE consists of a set of four principle executables:

- ``filter``: read a configurable set of filters and store a representation of them for later use.

- ``sedtolib``: read a configurable set of SED, and store the results into a binary library in a common format for later use.

- ``mag_gal``: use the preceding outputs to compute expected magnitudes and compute extinction corrections, applying different rules for galaxies, stars, or QSO objects.

- ``zphota``: performs chisquare minimization in order to derive photometric redshifts and other physical parameters.

These executables are configurable via a set of keywords, that can be passed at the command line or through a configuration file.
The list of these keywords can be found :doc:`here <keywords>`.

`filter` can often be called with just the config file

.. code-block:: bash

filter -c config_file.para

`sedtolib` will typically require the config and the object type. Updated library names which
overwrite the config can also be passed to it.

.. code-block:: bash

sedtolib -c config_file.para \
-t G \ # type G for Galaxies
--GAL_SED COSMOS_MOD.list \ # optional flag to overwrite
--GAL_LIB LIB_VISTA

`mag_gal` likewise can be run with just the config and type but is also highly customisable.

.. code-block:: bash

mag_gal -c config_file.para \
-t G \
--GAL_LIB_IN LIB_VISTA \
--GAL_LIB_OUT VISTA_COSMOS_FREE \
--MOD_EXTINC 18,26,26,33,26,33,26,33 \
--EXTINC_LAW SMC_prevot.dat,SB_calzetti.dat,SB_calzetti_bump1.dat,SB_calzetti_bump2.dat \
--EM_LINES EMP_UV \
--EM_DISPERSION 0.5,0.75,1.,1.5,2.

Finally `zphota` can be run similarly.

.. code-block:: bash

zphota -c config_file.para \
--CAT_IN COSMOS.in \
--CAT_OUT zphot_vista_adapt.out \
--ZPHOTLIB VISTA_COSMOS_FREE,ALLSTAR_COSMOS,QSO_COSMOS \
--ADD_EMLINES 0,100 \
--AUTO_ADAPT YES

Most of the optional flags that are sent to the command line arguments correspond to
keywords in the config file and effectively override them.

A more detailed example shell script which will run the COSMOS example can be found
`here <https://github.com/lephare-photoz/lephare/blob/main/docs/test_suite/test_suite.sh>`_.

This `example <https://github.com/lephare-photoz/lephare-data/blob/main/examples/README_full>`_
also shows some more advanced features which can be accessed via the command line.

Build only the C++ executables (historical method)
--------------------------------------------------

In addition to the Command Line Interface that is available immediately following
pip installation some users may want to use the original binaries to run lephare.
this might be useful if you don't want to depend on the Python interface or
as a means to solve installation issues with an unsuported operating system.
To do so requires making the C++ binaries using the MakeFile. It is possible to
install these binaries using either cmake or make. This is not the recommended
method but it remains a possibility.

The C++ part of LePhare has no external dependency beyond OpenMP and standard
compiling and linking libraries. The python part depends on cmake and several
packages from the python scientific ecosystem (see below).
In order to run make you will need an appropriate compiler for instance gcc on
a Mac which can be installed with ``brew install gcc``.

Follow the steps below to build with make which may require updating the
MakeFile for your system:

.. code-block:: bash

git clone https://gitlab.lam.fr/Galaxies/LEPHARE
cd LEPHARE
cd src/lib
make
#Add the current directory to the path
export PATH=$PATH:$(pwd)

Currently, with this method, the code just builds the executables in the *src/lib* directory.
To run code you would also need to download the additional data repository and
set the environment variables to the correct location

.. code-block:: bash

git clone https://github.com/lephare-photoz/LEPHARE-data.git
# LEPHAREDIR must point to the additional data repository
export LEPHAREDIR=$(pwd)/LEPHARE-data
# The LEPHAREWORK directory can be anywhere useful
mkdir lepharework
export LEPHAREWORK=$(pwd)/LEPHARE-work
3 changes: 3 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@
html_show_sourcelink = False
# Remove namespaces from class/method signatures
add_module_names = False
# Use the custom css
html_static_path = ["_static"]
html_css_files = ["custom.css"]

autoapi_type = "python"
autoapi_dirs = ["../src"]
Expand Down
Loading