Documentation read through

[raphaelshirley]

We are conducting a read through of the documentation and want to make a number of changes as we go. This issue will lead to a branch that will be used for this Documentation work.

[olivierilbert]

hello Rafael,

I am looking in detail to Intermediate_usage_notebook.ipynb. Here are my notes to improve few things:

In the cell "Update the config", I would remove the following lines. Not really useful in this context. I think that the one with the Z_STEP is sufficient to understand how to change a keyword.
To be removed:
# The following measurements will correspond to all filters. We could have an array of values for each.
"ERR_SCALE": "0.02",
"ERR_FACTOR": "1.5",
"FILTER_CALIB": "0",
"FILTER_FILE": "filter_test",

In "Download the required SEDs", I don't understand why do we need to specify the "additional_files". Is it not going to look for everything which is missing in the config? I don't understand what do you mean by "The additional extinction laws for galaxies are not in the principle config". So, how do we know the ones we need to specify and the ones which are downloaded automatically?

curl -s -o COSMOS.para https://github.com/lephare-photoz/lephare-data/blob/main/examples/COSMOS.para
The result COSMOS.para looks corrupted.

When using "config = lp.read_config("./COSMOS.para")" in "Update the config", the next "lp.data_retrieval.get_auxiliary_data(" fails.
I got this error message:
HTTPError: 404 Client Error: Not Found for url: https://raw.githubusercontent.com/lephare-photoz/lephare-data/main/STAR_SWIRE.list

"Getting new filters"
data.write(f"./filt/{f}", format="ascii", overwrite=True)
Is this line necessary to have the code running? A user could think it is the case. But the writting of the filters is done in "lp.write_output_filter". To avoid confusion, I would remove "data.write(f"./filt/{f}", format="ascii", overwrite=True)".

Update filter list
"SPEC_OUT": "YES", # We want to look at the model spectra
why is it here? I would remove.

"Run prepare"
strange to have the name CE using COSMOS templates. Maybe we can use by default something more neutral like LIB_GAL.

"Run process"
Do we have a simple way to run auto_adapt here ? I think we should explain this feature quickly because it is esential to get good results.

"Run process"
I got this warning after "output, pdfs, zgrid = lp.process(config, input_table)"
/home/ilbert/miniconda3/envs/lephare_github/lib/python3.12/site-packages/lephare/_photoz.py:34: UserWarning: Input line 35 contained no data and will not be counted towards max_rows=50000. This differs from the behaviour in NumPy <=1.22 which counted lines rather than rows. If desired, the previous behaviour can be achieved by using itertools.islice.
Please see the 1.23 release notes for an example on how to do this. If you wish to ignore this warning, use warnings.filterwarnings. This warning is expected to be removed in the future and is given only once per loadtxt call.
d = np.loadtxt(os.path.join(os.environ["LEPHAREDIR"], "alloutputkeys.txt"), dtype="str")

"Lower level functionality"
I am not sure that making a list of onesource object should be here. It should be in a different notebook.

[olivierilbert]

Here are my notes to improve few things in Example_full_run.ipynb:

"Example full run"

Remove the old text:
"Define the two environment variables necessary for Le Phare (directory of LEPHARE package, directory in which to store the libraries) and define the directory in which you run the code."

"Set up the parameters"

I would put a smaller step of 0.04 to have results which are nice, without takingf too much time.
keymap["Z_STEP"] = lp.keyword("Z_STEP", "0.04,0.,7.")

"Create filter library"

I think that these two lines should be put before, at the end of "Set up the parameters"
config_file = "./config_file.para"
lp.write_para_config(keymap, config_file)
config_file must be defined even if you skipped the building of the library to "Run the photoz"

"Run the photoz"

It is written that we are limited to the 100 first sources. But 10 is indicated in the following cells. I would put something large, like 1000.

t[:5] is duplicated.

A plot photo-z versus spec-z would be nice.

[olivierilbert]

Here are my note for the command lines section of the doc.

---

"Advanced Usage via Command Line Interface"

immeidately -> immediately

---

"Legacy code overview and main features"

This paragraph should be at the beginning of the page, at the beginning of "Advanced Usage via Command Line Interface".

Remove "compute extinction correction" from the sedtolib item. That's done in mag_gal. sedtolib convert all templates in a common format and unit.

. doesn't appear as a link.

---

"Build only the C++ executables with make (historical method)"
and "Legacy Installation"

I would merge this two sections and create a single one with this title "Build only the C++ executables (historical method)".

This should be the last paragraph of this page. We can present it as the historical method but also the method which doesn't require any python dependency in case of difficulty in the code installation (if working only with command lines).

The link is for gitlab, not github. TBC.

Also, it is required to have cloned the LEPHARE-data to make it works.
git clone https://github.com/lephare-photoz/LEPHARE-data.git

Also, I think that the two keywords need to be define by hand to make it work
export LEPHAREDIR=
export LEPHAREWORK=''

---

Build the C++ executables and the python module with cmake and setuptools

I think that this paragraph should be removed. That's a duplication of "Developer Guide" in "Getting Started". And the one in developper guide is more accurate (some typos in this one, and the "conda install cxx-compilers" missing).

So, I would simply removed this paragraph.

---

Quick Start

It is a duplication of what do you have at the beginning of the page. I would remove this paragraph, just making sure that there is not something missing in "Advanced Usage via Command Line Interface"

[olivierilbert]

Here are some comments for the page "getting started".

The page is really clear.

I would move everything which is in "Advanced Usage" within "Example Usage", just after the note. for me, that's necessary to have such understanding to run the code (not advanced usage).

In "Auxiliary Data and the Environment Variables", it would be nice to include the git clone line to be used.

[olivierilbert]

Here are some changes for the keywords. I review all keywords and try to take the best explanation between this version and the detailed documentation.

---

sedtolib

GAL_SED/QSO_SED/STAR_SED
Absolute or relative to $LEPHAREWORK: to be removed. That's the absolute path.

GAL_FSCALE/QSO_SCALE/STAR_SCALE
Arbitrary flux scale to be applied to each SED flux in the list

SEL_AGE
values: string
filename of the file containing the ages to consider (in Gyr)

AGE_RANGE
age range to be considered (in Gyr)

---

mag_gal

COSMOLOGY
cosmological parameters

EXTINC_LAW
Extinction laws stored in $LEPHAREDIR/ext/. Could include several files separated by comma.

GAL_LIB_IN / QSO_LIB_IN / STAR_LIB_IN
value: string
Name of the GALAXY/QSO/STAR binary library (with no extension). Files must exist in $LEPHAREWORK/lib_bin/ and have been created by a sedtolib run.

GAL_LIB_OUT / QSO_LIB_OUT / STAR_LIB_OUT
value: string
Name of the output library (with no extension) storing predicted fluxes/magnitude/k-corrections.
Files .bin and .doc are saved in $LEPHAREWORK/lib_mag/

LIB_ASCII
-> check where it is stored now.

ADD_DUSTEM
value: NO[def] or YES
Add the dust emission in templates when missing (using energy balance).

---

zphota

It would be nice to split this table in smaller pieces, according to what is done in the detailed description:
https://lephare.readthedocs.io/en/latest/original.html#introduction
It would make the user life easier to find the relevant keyword.

CAT_IN
value: string
aspace -> a space

ERR_FACTOR
Scaling factor to the errors (in flux). Only a single value applied to all filters.

BD_SCALE/ GLB_CONTEXT / FORB_CONTEXT
sum of $2^i$ -> sum of 2^i starting at i=0

MASS_SCALE
Prior: allowed range in mass -> Prior: allowed range in log10(mass)

MAG_ABS
Prior: Absolute magnitude range allowed for the GAL library [0,0-def]

MAG_ABS_QSO
Prior: Absolute magnitude range allowed for the QSO library [0,0-def]

NZ_PRIOR
N(z) prior as function of i-band. The i-band number should be given in input (starting filter numbering at 1). The second number indicates which band to use if first undefined. Negative value means no prior.

ZFIX
Fixed redshift with the spec-z value (as defined in CAT_TYPE LONG)

EXTERNALZ_FILE
value: string
Name of an external file. Use the spec-z from an external file (format Id,zs) to fix the redshift.

DZ_WIN
Window function for 2nd peak search in L(z) (minimal distance in dz from the 1st peak)

SPEC_OUT
Output files with Gal/Star/QSO spectra (one file per object)
(if YES: can take a lot of disk space !)

CHI2_OUT
Output files with the chi2 for the full library (one file per object)
(if YES: can take a lot of disk space !)

PDZ_TYPE
value: BAY_ZG[def] or/and BAY_ZQ,MIN_ZG,MIN_ZQ,MASS,SFR,SSFR,AGE
PDZ in output [def-BAY]. BAY_ZG sum all probabilities at a given z.
MIN_ZG takes ex p(-chi2_min/2) at a each z.

MABS_METHOD
Method used for absolute magnitudes in each filter
0 (default): mag(filter) -> Mabs(filter)
1 : mag(best filter)-> Mabs(filter)
2 : mag(fixed filter with MABS_REF)-> Mabs(filter)
3 : best SED -> Mabs(filter)
4 : MABS(filter) derives according to a fixed filter in a fixed redshift interval as given by MABS_FILT and MABS_ZBIN

Z_METHOD
Compute the absolute magnitude at a given redshift solution maginalised over all models (ML) or for the best model which minimize the chi2 (BEST)

MABS_REF
Filter in observed frame used to derive all the Mabs if method=2

ADDITIONAL_MAG
value: string
Name of file compiling several filters (in $LEPHAREWORK/filt, created by filter)to derive Mabs in additional filters

M_REF and RF_COLORS: never tested seriously. Not sure to make them appearing. M_REF is really a bad name since redondant with above.

APPLY_SYSSHIFT
Apply systematic shifts in each bands (convention: values in magnitude to be substracted to the observed magnitudes). Number of values must correspond to the number of filters.

ADAPT_BAND
Reference band for the selection in magnitude (start at 1)

RM _DISCREPENT_BD
value: 200[def]
Threshold in chi2 to stop removing bands. Remove 2 bands max, stop when below this chi2 threshold.

Z_RANGE
value: 0.,99.[def]
Z min and max allowed in the GAL library

EBV_RANGE
0,9[def]
E(B-V) min and max allowed in the GAL library

I think I would remove the full section:
"Not implemented in latest version"
Some are indeed implemented, some were never tested in the frotran version.

[raphaelshirley]

I have implemented most of these. We can talk today about a couple of points. The tables of keywords should probably be alphabetized. By wrapping the descriptions they are more readable now. I can't see an obvious way to break the zphota table. Is there a logical way to break it in two?