deploy: 1d140e7

LSSTDESC · Nov 22, 2024 · 4e34a32 · 4e34a32
commit 4e34a32
Show file tree

Hide file tree

Showing 101 changed files with 9,621 additions and 0 deletions.
diff --git a/.buildinfo b/.buildinfo
@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file records the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 9e59515d8435e86e1709778b4229d9c8
+tags: 645f666f9bcd5a90fca523b33c5a78b7
diff --git a/.doctrees/contact.doctree b/.doctrees/contact.doctree
diff --git a/.doctrees/environment.pickle b/.doctrees/environment.pickle
diff --git a/.doctrees/index.doctree b/.doctrees/index.doctree
diff --git a/.doctrees/installation.doctree b/.doctrees/installation.doctree
diff --git a/.doctrees/ops-rehearsals-3-4/SSO.doctree b/.doctrees/ops-rehearsals-3-4/SSO.doctree
diff --git a/.doctrees/ops-rehearsals-3-4/UW_stars.doctree b/.doctrees/ops-rehearsals-3-4/UW_stars.doctree
diff --git a/.doctrees/ops-rehearsals-3-4/galaxies.doctree b/.doctrees/ops-rehearsals-3-4/galaxies.doctree
diff --git a/.doctrees/reference_python.doctree b/.doctrees/reference_python.doctree
diff --git a/.doctrees/roman-rubin-1.1.2/UW_stars.doctree b/.doctrees/roman-rubin-1.1.2/UW_stars.doctree
diff --git a/.doctrees/roman-rubin-1.1.2/diffsky_galaxy.doctree b/.doctrees/roman-rubin-1.1.2/diffsky_galaxy.doctree
diff --git a/.doctrees/roman-rubin-1.1.2/snana.doctree b/.doctrees/roman-rubin-1.1.2/snana.doctree
diff --git a/.doctrees/usage_create.doctree b/.doctrees/usage_create.doctree
diff --git a/.doctrees/usage_read.doctree b/.doctrees/usage_read.doctree
diff --git a/.nojekyll b/.nojekyll
diff --git a/_sources/contact.rst.txt b/_sources/contact.rst.txt
@@ -0,0 +1,12 @@
+Contact
+=======
+
+If you need to contact someone directly for assistance we are happy to help!
+
+- Admin, developer: **Joanne Bogart** (`@JoanneBogart <https://www.github.com/JoanneBogart>`__)
+- Developer: **Jim Chiang** (`@jchiang87 <https://www.github.com/jchiang87>`__)
+
+For any bugs, or suggestions for additional features within the DESC data
+management software, please raise an issue via the GitHub repository
+(`https://github.com/LSSTDESC/skyCatalogs/issues
+<https://github.com/LSSTDESC/skyCatalogs/issues>`__).
diff --git a/_sources/index.rst.txt b/_sources/index.rst.txt
@@ -0,0 +1,55 @@
+.. skyCatalogs documentation master file, created by
+   sphinx-quickstart on Fri Apr 12 13:47:18 2024.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+=======================================================
+skyCatalogs: provides API for simulated object catalogs
+=======================================================
+skyCatalogs both creates simulated object catalogs (from precursor catalogs)
+and provides a uniform API to the catalogs it makes and to certain other
+catalogs. These catalogs may be used as inputs to image simulation (originally
+only for the Rubin Observatory; since also used in simulations for the
+Roman Observatory) and as truth catalogs.
+
+This documentation covers installation and usage instructions for the package.
+
+The source code of *skyCatalogs* is hosted at
+https://github.com/LSSTDESC/skyCatalogs.
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   installation
+   usage_read
+   usage_create
+   ops-rehearsals-3-4/galaxies
+   ops-rehearsals-3-4/UW_stars
+   ops-rehearsals-3-4/SSO
+   roman-rubin-1.1.2/diffsky_galaxy
+   roman-rubin-1.1.2/UW_stars
+   roman-rubin-1.1.2/snana
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Reference:
+   :hidden:
+
+   reeference_python
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contact:
+   :hidden:
+
+   contact
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
diff --git a/_sources/installation.rst.txt b/_sources/installation.rst.txt
@@ -0,0 +1,107 @@
+Installation Instructions
+=========================
+
+.. note::
+
+   **Prerequisites**:
+
+   To use **skyCatalogs 1.7.x**
+
+   all that is required is a reasonably current version of the  `LSST science pipelines <https://pipelines.lsst.io/>`_ .  See installation instructions below.
+
+.. note::
+
+   **Use with imSim**:
+
+   If you intend to use skyCatalogs with imSim, you should follow the `imSim installation instructions <https://lsstdesc.org/imSim/install.html/>`_ . In that case, you need proceed no further with the instructions here; you're all set.  The remainder of these notes is largely extracted from the imSim instructions, omitting anything not required for skyCatalogs.
+
+Installing LSST science pipelines
+---------------------------------
+
+There are several methods of installation.  Only the simplest (using a prebuilt cvmfs version) is described here.  For other methods, see the imSim installation instructions.
+
+If you are working at the USDF (Rubin Project computing) or at NERSC (DESC computing), perhaps the easiest way to setup and use *skyCatalogs* is to rely on the prebuilt versions of the pipelines contained in the cvmfs distribution which is installed there.  This solution is also appropriate for personal laptops and university/lab based computing systems if you are able to install the *cvmfs* system.
+
+The `CernVM file system <https://cvmfs.readthedocs.io/>`_  (cvmfs) is a distributed read-only file system developed at CERN for reliable low-maintenance world-wide software distribution.  LSST-France distributes weekly builds of the Rubin science pipelines for both Linux and MacOS.  Details and installation instructions can  be found at `sw.lsst.eu <https://sw.lsst.eu/index.html>`_ .  The distribution includes conda and skyCatalogs dependencies from conda-forge along with the science pipelines.
+
+Load and setup the science pipelines
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+First you need to setup the science pipelines.  This involves sourcing a setup file and then using the Rubin *eups* commands to set them up.
+
+.. note::
+
+   Version  ``w_2024_20`` or later of the science pipelines is recommended. This will guarantee other dependencies of skyCatalogs, such as GalSim, are new enough.
+
+   Also note: the cvmfs distribution is a read-only distribution.  This means you cannot add packages to the included conda environment and packages you install via *pip* will be installed in the user area.  If you need a *conda*  environment you will need to use a different installation method.
+
+Source the appropriate setup script (note the -ext in the name) and then setup the distribution (if you are on MacOS use darwin-x86_64 instead of linux-x86_64).
+
+.. code-block:: sh
+
+   source /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/w_2024_20/loadLSST-ext.bash
+   setup lsst_distrib
+
+
+Install needed data files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Now go to where you would like to install *skyCatalogs* and download some needed data files (you will only need to do this once).
+
+.. code-block:: sh
+
+   mkdir -p rubin_sim_data/sims_sed_library
+   curl https://s3df.slac.stanford.edu/groups/rubin/static/sim-data/rubin_sim_data/throughputs_2023_09_07.tgz | tar -C rubin_sim_data -xz
+   curl https://s3df.slac.stanford.edu/groups/rubin/static/sim-data/sed_library/seds_170124.tar.gz  | tar -C rubin_sim_data/sims_sed_library -xz
+
+Set up skyCatalogs itself
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Clone the skyCatalogs package from GitHub:
+
+.. code-block:: sh
+
+   git clone https://github.com/LSSTDESC/skyCatalogs
+
+at this point if you would only like to use *skyCatalogs* you can  ``pip install skyCatalog/`` however we instead suggest using the *eups* tool to simply setup the package for use without installing it. This will allow you to edit the package in place, use multiple versions, change branches etc. You should definitely do this if you plan to do any *skyCatalogs* development.
+
+If you do not intend to do any development you may choose instead to clone the most recent release tag.  As of Oct., 2024 this is 1.7.0rc4
+
+.. code-block:: sh
+
+   git clone https://github.com/LSSTDESC/skyCatalogs.git --branch 1.7.0rc4
+
+
+Setup and Use *skyCatalogs*
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This setup step should be repeated for each new session.  Here is a ``skycatalogs-setup.sh`` file you can use before each session
+
+.. code-block:: sh
+
+   source /cvmfs/...            # as above
+   setup lsst_distrib
+
+   export SKYCATALOGS_HOME=*PUT YOUR INSTALL DIRECTORY HERE*
+   export RUBIN_SIM_DATA_DIR=$SKYCATALOGS_HOME/rubin_sim_data
+   export SIMS_SED_LIBRARY_DIR=$SKYCATALOGS_HOME/rubin_sim_data/sims_sed_library
+
+   setup -k -r $SKYCATALOGS_HOME/skyCatalogs
+
+You should now be able to import the code you need from the skyCatalogs package, e.g.
+
+.. code-block:: python
+
+   from skycatalogs.skyCatalogs import open_catalog
+   from skycatalogs.utils.shapes import Disk
+
+   skycatalog_root = "path_to/skycatalog_files"  # folder containing catalog
+   config_file = "some_folder/skyCatalog.yaml"
+
+   cat = open_catalog(config_file, skycatalog_root=skycatalog_root)
+
+   # define disk at ra, dec = 45.0, -9.0 of radius 100 arcseconds
+   disk = disk(45.0, -9.0, 100.0)
+
+   # get galaxies and stars in the region
+   objects = cat.get_objects_by_region(disk, obj_type_set={'galaxy', 'star'})
diff --git a/_sources/ops-rehearsals-3-4/SSO.rst.txt b/_sources/ops-rehearsals-3-4/SSO.rst.txt
@@ -0,0 +1,39 @@
++++++++++++++++++++++++++++++++++++++++++
+SSO quantities for ops-rehearsals 3 and 4
++++++++++++++++++++++++++++++++++++++++++
+Data are partitioned by (nside=32, ring ordering) healpixel. For each pixel
+there is a so-called "main file" and a flux file. The latter contains
+for each object only fluxes for lsst bands, the object id and the time (mjd)
+of the observation. The latter two fields may be used to join with the
+main file.
+
+SSO main file
+-------------
+
+========================  ============   ==============  ===========================
+Name                      Datatype       Units           Description
+========================  ============   ==============  ===========================
+id                        string         N/A             Unique object identifier
+mjd                       double         days            Time of observation
+ra                        double         degrees         Object right ascension
+dec                       double         degrees         Object declination
+trailed_source_mag        double         N/A             Mag. normalization for SED
+ra_rate                   double         deg/day         RA rate of change * cos(dec)
+dec_rate                  double         deg/day         Declination rate of change
+========================  ============   ==============  ===========================
+
+SSO flux file
+----------------
+
+=============   =========   ================  ================================
+Name            Datatype    Units             Description
+=============   =========   ================  ================================
+id              string      N/A               Unique object identifier
+mjd             double      days              Time of observation (Julian date)
+lsst_flux_u     float       photons/sec/cm^2  Flux integrated over lsst u-band
+lsst_flux_g     float       photons/sec/cm^2  Flux integrated over lsst g-band
+lsst_flux_r     float       photons/sec/cm^2  Flux integrated over lsst r-band
+lsst_flux_i     float       photons/sec/cm^2  Flux integrated over lsst i-band
+lsst_flux_z     float       photons/sec/cm^2  Flux integrated over lsst z-band
+lsst_flux_y     float       photons/sec/cm^2  Flux integrated over lsst y-band
+=============   =========   ================  ================================
diff --git a/_sources/ops-rehearsals-3-4/UW_stars.rst.txt b/_sources/ops-rehearsals-3-4/UW_stars.rst.txt
@@ -0,0 +1,59 @@
+++++++++++++++++++++++++++++++++++++++++++
+Star quantities for ops-rehearsals 3 and 4
+++++++++++++++++++++++++++++++++++++++++++
+Data are partitioned by (nside=32, ring ordering) healpixel. For each pixel
+there is a so-called "main file" (with information largely coming from the
+UW star catalog) and a flux file. The latter contains for each object only
+fluxes for lsst bands and the object id so it can be joined with the main
+file. Both are parquet files.
+
+Star main file
+----------------
+Note the same schema may be used for SNe. Some fields are unused for stars.
+
+========================  ============   ==========  =========================
+Name                      Datatype       Units       Description
+========================  ============   ==========  =========================
+object_type               string         N/A         In practice always 'star'
+id                        string         N/A         Unique object identifier
+ra                        double         degrees     Object right ascension
+dec                       double         degrees     Object declination
+host_galaxy_id            int64          N/A         Unused for stars
+magnorm                   double         N/A         To be applied to SED
+sed_filepath              string         N/A         Path to object's SED file
+                                                     relative to env variable
+                                                     SIMS_SED_LIBRARY_DIR
+MW_rv                     float          N/A         Extinction parameter
+MW_av                     float          N/A         Extinction parameter
+                                                     from F19 dust map
+mura                      double         milarcsec   RA proper motion
+                                         per year
+mudec                     double         milarcsec   Declin. proper motion
+                                         per year
+radial_velocity           double         km/sec      Radial velocity
+parallax                  double         milarcsec   Parallax
+variability_model         string         N/A         Unused for stars
+salt2_params              int32          N/A         Unused for stars
+is_variable               boolean        N/A         If true, the star has
+                                                     sinusoidal variability
+                                                     in magnitude
+period                    double         days        period of variability
+mag_amplitude             double         N/A         amplitude of magnitude
+                                                     variability
+phase                     double         radians     phase of variability
+========================  ============   ==========  =========================
+
+Star flux file
+----------------
+
+=============   =========   ================  ================================
+Name            Datatype    Units             Description
+=============   =========   ================  ================================
+id              string      N/A               Unique object identifier
+lsst_flux_u     float       photons/sec/cm^2  Flux integrated over lsst u-band
+lsst_flux_g     float       photons/sec/cm^2  Flux integrated over lsst g-band
+lsst_flux_r     float       photons/sec/cm^2  Flux integrated over lsst r-band
+lsst_flux_i     float       photons/sec/cm^2  Flux integrated over lsst i-band
+lsst_flux_z     float       photons/sec/cm^2  Flux integrated over lsst z-band
+lsst_flux_y     float       photons/sec/cm^2  Flux integrated over lsst y-band
+=============   =========   ================  ================================
diff --git a/_sources/ops-rehearsals-3-4/galaxies.rst.txt b/_sources/ops-rehearsals-3-4/galaxies.rst.txt
@@ -0,0 +1,65 @@
+++++++++++++++++++++++++++++++++++++++++++++
+Galaxy quantities for ops-rehearsals 3 and 4
+++++++++++++++++++++++++++++++++++++++++++++
+Data are partitioned by (nside=32, ring ordering) healpixel. For each pixel
+there is a so-called "main file" (with information largely coming from the
+cosmodc2 galaxy catalog) and a flux file. The latter contains
+for each object only fluxes for lsst bands and the object id so it can be
+joined with the main file. Both are parquet files. The main file can be
+rather large (over 3 Gbytes) so, depending on your application, you may
+want to iterate over row groups rather than reading it all in at once.
+
+Galaxy main file
+----------------
+
+=============================  ============   ==============  ========================
+Name                           Datatype       Units           Description
+=============================  ============   ==============  ========================
+galaxy_id                      int64          N/A             Unique object identifier
+ra                             double         degrees         Object right ascension
+dec                            double         degrees         Object declination
+redshift                       double         N/A             Cosmological redshift
+                                                              with line-of-sight motion
+redshiftHubble                 double         N/A             Cosmological redshift
+peculiarVelocity               double         km/sec          Peculiar velocity
+shear1                         double         N/A             Shear (gamma) component 1,
+                                                              treecorr/Galsim convention
+shear2                         double         N/A             Shear (gamma) component 2,
+                                                              treecorr/Galsim convention
+convergence                    double         N/A             Convergence (kappa)
+spheroidHalfLightRadiusArcsec  float          arcsec          Bulge component half-light
+                                                              radius
+diskHalfLightRadiusArcsec      float          arcsec          Disk component half-light
+                                                              radius
+diskEllipticity1               double         N/A             Ellipticity component 1
+                                                              for disk, not lensed
+diskEllipticity2               double         N/A             Ellipticity component 2
+                                                              for disk, not lensed
+spheroidEllipticity1           double         N/A             Ellipticity component 1
+                                                              for bulge, not lensed
+spheroidEllipticity2           double         N/A             Ellipticity component 2
+                                                              for bulge, not lensed
+um_source_galaxy_obs_sm        float          solar mass      stellar mass
+                                              (h=0.7)
+MW_rv                          float          N/A             Extinction parameter
+MW_av                          float          N/A             Extinction parameter
+                                                              from F19 dust map
+=============================  ============   ==============  ========================
+
+NOTE: For tophat definitions (start and width in units of angstroms) see yaml
+config file associated with the data.
+
+Galaxy flux file
+----------------
+
+=============   =========   ================  ================================
+Name            Datatype    Units             Description
+=============   =========   ================  ================================
+galaxy_id       int64       N/A               Unique object identifier
+lsst_flux_u     float       photons/sec/cm^2  Flux integrated over lsst u-band
+lsst_flux_g     float       photons/sec/cm^2  Flux integrated over lsst g-band
+lsst_flux_r     float       photons/sec/cm^2  Flux integrated over lsst r-band
+lsst_flux_i     float       photons/sec/cm^2  Flux integrated over lsst i-band
+lsst_flux_z     float       photons/sec/cm^2  Flux integrated over lsst z-band
+lsst_flux_y     float       photons/sec/cm^2  Flux integrated over lsst y-band
+=============   =========   ================  ================================