Rejection samp (#4379)

* unexpected events

* added temp-volume in cl

* added rejection sampling

* added indexing of mchirp in fit_param

* added compensation factor

* added corrections

* added more corrections

* added random number generator seed

* added mchirp instead of log(mchirp) in rejection samp

* Use probabilities to compensate downsampled kde

* Try to deal with flaky failures when apt-getting (#4384)

* Implement pre-cuts in fit_over_multiparam for efficiency (#4374)

* Make some efficiency savings in pycbc_fit_sngls_over_multiparam

* Missing logging line in once case

* Add in progress reporting, in case of silent failure

* I thought we needed some leeway, but we don't

* TD comments

* Simplify comment

* Migrate to new gwdatafind API (#4383)

* Migrate to new gwdatafind API

* CC and one regression fix

* One more style change

* Don't need these options now

* Revert "Don't need these options now"

This reverts commit 6f2ed1b.

* Add a Fisher distribution for sky locations (#4209)

* Update transforms.py

Adding new_z_to_euler and rotate_euler from pylal.sphericalutils

* Update sky_location.py

Added new class FisherDist for drawing from sky locations (in ra, dec)

* Update sky_location.py

Included help message for fisherDist

* Update sky_location.py

* Code climate issues

* Code climate issues

* Code-climate changes

* Code climate issues

* Code climate issues

* Update transforms.py

changed lal.PI to numpy.pi in new_z_to_euler

* Update transforms.py

minor changes

* Update sky_location.py

updated docstring, did changes to make the code look more simple

* Update sky_location.py

typos corrected

* Update sky_location.py

code climate changes

* Update sky_location.py

* Update sky_location.py

Included the definitions rvs_polaz and rvs_radec

* Update transforms.py

Included the definitions decra2polaz and polaz2decra

* Update sky_location.py

* Update transforms.py

code climate issues fixed

* Update sky_location.py

code climate issues fixed

* Update sky_location.py

code climate issues

* Update sky_location.py

fixed code climate issues

* Update sky_location.py

fixed code climate issues

* Update sky_location.py

* Update pycbc/distributions/sky_location.py

Co-authored-by: Tito Dal Canton <[email protected]>

* Update sky_location.py

* Update sky_location.py

* Update transforms.py

modified polaz2decra to polaz2radec

* Update sky_location.py

* Update sky_location.py

Forgotten polaz2decra --> polaz2radec

* Update sky_location.py

Code climate fixes

* Update sky_location.py

Updated the code for checking the effectiveness of running from config file

* Create sky_location_old.py

* modified to include from config

* Modified to run from config file

* Added Fisher distribution to the lista of available distributions

* Delete sky_location_old.py

* Update sky_location.py

* Renamed Fisher to FisherSky

* Updated the name to FisherSky

* changed Fisher to FisherSky in __all__

* removed decra2polaz and polaz2radec to conversions

* Included def decra2polaz, polaz2radec in line 859

* Added angle_unit option

* removed decra2polaz and polaz2decra

* made carresction

* made corrections

* Corrected the typing error while removing recra2polaz

* Modified angle_unit and updated init file

* Empty line on 852 retrieved

* corrected

* Empty line on852 retrieved

* Fixed code climate issues

* Sigma conversion taken care of

* Fixed code climate issues

* Update sky_location.py

Codeclimate

* Update test_distributions.py

Adding fisher_sky to the list of distributions that do not undergo the distribution unittests.

* Update sky_location.py

Syntax fixed.

* Update sky_location.py complying to codeclimate

* Refactor using scipy's Rotation

* Improve docstring and make warning stricter

* Update pycbc/distributions/sky_location.py

Co-authored-by: Francesco Pannarale <[email protected]>

* Update pycbc/distributions/sky_location.py

right angle to correct angle

Co-authored-by: Francesco Pannarale <[email protected]>

* Update sky_location.py

logging issue solved

---------

Co-authored-by: Francesco Pannarale <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: P Prasia <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>

* add in several missing marginalization combinations (#4376)

* allow polarization marginalization with earth rotation

* add in missing marg combos

* fixes

* fixes

* fixes

* cleanup

* cc

* cc

* Try to fix CI tests (#4394)

* Switch FFTW/MKL order of preference

* Swap backend order and force gomp import

* FIx typo

* Do need openmpi in tox.ini

* Fix bug in MKL error function

* Force FFTW on this one.

* CC

* Requested Josh changes

* Add bank corner plot (#4339)

* Add corner plotting script for template banks

* Comment where premerger duration has been subtracted from template duration in bank_conversions

* add premerger_duration to bank_conversions.py

* Move premerger duration into bank_conversions

* Use bank's approximant

* Dont plot signular values as default

* CC complaints

* docstring

* TDC comments

* Keep approximant kwarg as default, TDC comments 2

* Added an OSError exception to MKL check (#4390)

* Add a possibility for defaultvalues sections in config files (#4385)

* Add a possibility for defaultvalues sections in config files

* do not allow [section] and [section-defaultvalues] options

* defaultvalues for top-level sections only

* underscore on add_ini_opts

* Remove default sections from get_subsections (#4399)

* Update setup-python action to v4 (#4400)

* Replace LOSC with GWOSC (#4401)

* Replace LOSC with GWOSC

* Fix leftover Python 2-ism

* Try to improve codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Exclude IFO sections when looping through the get_subsections (#4402)

* Exclude IFO sections when looping through the get_subsections for plots

* Add an example whihc would have previously failed into the search config

* Offline singles in workflow (#4386)

* Add single-detector options into workflow

* Fix a couple of isses which make results pages weird

* fix

* fix gen.sh to use correct workflow generator

* revert change made in error

* A couple of points which state coincident when it may not be

---------

Co-authored-by: Thomas Dent <[email protected]>
Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Gareth S Cabourn Davies <[email protected]>
Co-authored-by: Prasia-Pankunni <[email protected]>
Co-authored-by: Francesco Pannarale <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: P Prasia <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: Alex Nitz <[email protected]>
Co-authored-by: Arthur Tolley <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>

bin/all_sky_search/pycbc_template_kde_calc

@@ -49,8 +49,12 @@ parser.add_argument('--bw-grid', type=float, nargs='+',
                     help='Grid of choices of global bandwidth')
 parser.add_argument('--extra-cpt-fraction', type=float,
                     help='Fraction of the extra component in the signal density')
-parser.add_argument('--temp-volume', type=float, 
+parser.add_argument('--temp-volume', type=float,
                     help='Volume covered by the template bank')
+parser.add_argument('--seed', type=int,
+                    help='Random number generator seed')
+parser.add_argument('--mchirp-downsample-power', type=float,
+                    help='Exponent value for the power law distribution')
 parser.add_argument('--verbose', action='store_true')
 args = parser.parse_args()
 init_logging(verbose=args.verbose, format='%(asctime)s %(message)s')
@@ -133,7 +137,7 @@ def optimizedparam(sampleval, bwgrid, alphagrid, nfold=2):
 
 # Obtaining template parameters
 temp_file = h5py.File(args.template_file, 'r')
-mass1 = temp_file['mass1']
+mass1 = temp_file['mass1'][:]
 tid = numpy.arange(len(mass1))  # Array of template ids
 mass_spin = triggers.get_mass_spin(temp_file, tid)
 
@@ -162,13 +166,47 @@ temp_samples = numpy.vstack((template_pars)).T
 
 
 if args.make_template_kde:
-    logging.info('Starting optimization of template KDE parameters')
-    optbw, optalpha = optimizedparam(temp_samples, alphagrid=args.alpha_grid,
-                                     bwgrid=args.bw_grid, nfold=args.nfold_template)
-    logging.info('Bandwidth %.4f, alpha %.2f' % (optbw, optalpha))
-    logging.info('Evaluating template KDE')
-    template_kde = kde_awkde(temp_samples, temp_samples, alp=optalpha,
-                             gl_bandwidth=optbw)
+
+    # Rejection Sampling
+    if args.mchirp_downsample_power is not None:
+        logging.info('Downsampling with mchirp power '
+                     f'{args.mchirp_downsample_power}')
+        f_dest.attrs['mchirp_downsample_power'] = args.mchirp_downsample_power
+        try:
+            mchirp_index = args.fit_param.index('mchirp')
+        except:
+            raise ValueError("mchirp does not exist in args.fit_param")
+        mc_vals = template_pars[mchirp_index]
+        if args.log_param[mchirp_index] is True:
+            mc_vals = numpy.exp(mc_vals)
+        power_vals = mc_vals ** args.mchirp_downsample_power
+        probabilities = power_vals / numpy.max(power_vals)
+        if args.seed is not None:
+            numpy.random.seed(args.seed)
+        rand_nums = numpy.random.uniform(0, 1, len(mass1))
+        ind = rand_nums < probabilities
+        logging.info(f'{ind.sum()} templates after downsampling')
+        kde_train_samples = temp_samples[ind]
+        f_dest.create_dataset("kde_train_samples", data=kde_train_samples)
+        logging.info('Starting optimization of template KDE parameters')
+        optbw, optalpha = optimizedparam(kde_train_samples, alphagrid=args.alpha_grid,
+                                         bwgrid=args.bw_grid, nfold=args.nfold_template)
+        logging.info('Bandwidth %.4f, alpha %.2f' % (optbw, optalpha))
+        logging.info('Evaluating template KDE')
+        template_kde = kde_awkde(kde_train_samples, temp_samples, alp=optalpha,
+                                 gl_bandwidth=optbw)
+        # Compensation factor for downsampling of templates
+        template_kde *= 1. / probabilities
+
+    else:
+        logging.info('Starting optimization of template KDE parameters')
+        optbw, optalpha = optimizedparam(temp_samples, alphagrid=args.alpha_grid,
+                                         bwgrid=args.bw_grid, nfold=args.nfold_template)
+        logging.info('Bandwidth %.4f, alpha %.2f' % (optbw, optalpha))
+        logging.info('Evaluating template KDE')
+        template_kde = kde_awkde(temp_samples, temp_samples, alp=optalpha,
+                                 gl_bandwidth=optbw)
+
     f_dest.create_dataset("data_kde", data=template_kde)
     f_dest.attrs['stat'] = "template-kde_file"
     f_dest.attrs['template-file'] = args.template_file