/issue/637/clusterensemble_dc2: Codes and example notebook for the use of the ClusterEnsemble object with LSST DESC DC2 data

[payerne]

Associated to issue 637.

This branch is dedicated to the creation of an example notebook for the use of the ClusterEnsemble object with cosmoDC2 data. We consider using a sub-sample of redMaPPer-detected cluster (\sim 400 objects) and we estimate their lensing signal using cosmoDC2 background galaxies (no shape measurement and no photometric redshift).

With this notebook we provide source code to extract background source catalogs and estimate individual lensing profiles. The example notebook loads the individual profiles and makes use of the ClusterEnsemble object to estimate the cluster sample stacked lensing profile and covariance matrix.

We also provide individual lensing profiles already extracted, since it takes some time to do it.

In practice, the branch has new files:

1. _utils_cosmoDC2.py has a few useful functions to handle cosmoDC2 data, especially for the extraction of lensing catalogs using Qserv or GCRCatalogs at CC-IN2P3.
2. extract_cluster_catalog_redMaPPer.py to extract the redMaPPer cluster catalog with GCRCatalogs.
3. extract_sources_and_compute_ind_profile_cosmoDC2_per_split.py to run the extract of lensing catalogs behind redMaPPer clusters.
4. jobs_extract_sources_cosmoDC2.py that writes and runs parallel jobs to speed up the extraction of lensing catalogs for a stack of 450 clusters (around 30 minutes).
5. DC2_data/... gathers the different DC2 datasets (redMaPPer cluster catalog and individual lensing profiles).
6. DC2_redMaPPer_clusterensemble_object.ipynb is a notebook to use the ClusterEnsemble to estimate stacked profiles and covariances, and to do some plots.

Some figures from this notebook were added to the CLMMv2 paper.

[coveralls]

coverage: 100.0%. remained the same
when pulling 5908f24 on issue/637/clusterensemble_dc2
into cfa7f11 on main.

[m-aguena]

I removed DC2_data from repo, and made a copy at CC:

/sps/lsst/groups/clusters/CLMM/DC2_data/

and at NERSC:

/global/cfs/projectdirs/lsst/groups/CL/CLMM/DC2_data/

[payerne]

• I change the path that was used to load the redmapper cluster catalog and individual cluster lensing profiles to
  -- path_DC2_data_ccin2p3 = '/sps/lsst/groups/clusters/CLMM/DC2_data/'
  -- path_DC2_data_nersc = '/global/cfs/projectdirs/lsst/groups/CL/CLMM/DC2_data/'
  to be chosen by the user.
• I re-defined the small cluster sample by selecting randomly 100 clusters in the original sample of 450 clusters.
• I computed the correlation matrix, to be added in the CLMM v2.0 draft.

[payerne]

I tried to solve the issue "All checks have failed" by merging the branch with the latest vesting of main, without success. It seems to come from the installation of CCL.

[hsinfan1996]

@payerne It seems to me that the problem is related to swig 4.3.0. I will take a look.