Skip to content

Commit

Permalink
Update feature branch (#4444)
Browse files Browse the repository at this point in the history
* Try to fix CI tests (#4394)

* Switch FFTW/MKL order of preference

* Swap backend order and force gomp import

* FIx typo

* Do need openmpi in tox.ini

* Fix bug in MKL error function

* Force FFTW on this one.

* CC

* Requested Josh changes

* Add bank corner plot (#4339)

* Add corner plotting script for template banks

* Comment where premerger duration has been subtracted from template duration in bank_conversions

* add premerger_duration to bank_conversions.py

* Move premerger duration into bank_conversions

* Use bank's approximant

* Dont plot signular values as default

* CC complaints

* docstring

* TDC comments

* Keep approximant kwarg as default, TDC comments 2

* Added an OSError exception to MKL check (#4390)

* Add a possibility for defaultvalues sections in config files (#4385)

* Add a possibility for defaultvalues sections in config files

* do not allow [section] and [section-defaultvalues] options

* defaultvalues for top-level sections only

* underscore on add_ini_opts

* Remove default sections from get_subsections (#4399)

* Update setup-python action to v4 (#4400)

* Replace LOSC with GWOSC (#4401)

* Replace LOSC with GWOSC

* Fix leftover Python 2-ism

* Try to improve codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Exclude IFO sections when looping through the get_subsections (#4402)

* Exclude IFO sections when looping through the get_subsections for plots

* Add an example whihc would have previously failed into the search config

* Offline singles in workflow (#4386)

* Add single-detector options into workflow

* Fix a couple of isses which make results pages weird

* fix

* fix gen.sh to use correct workflow generator

* revert change made in error

* A couple of points which state coincident when it may not be

* Rejection samp (#4379)

* unexpected events

* added temp-volume in cl

* added rejection sampling

* added indexing of mchirp in fit_param

* added compensation factor

* added corrections

* added more corrections

* added random number generator seed

* added mchirp instead of log(mchirp) in rejection samp

* Use probabilities to compensate downsampled kde

* Try to deal with flaky failures when apt-getting (#4384)

* Implement pre-cuts in fit_over_multiparam for efficiency (#4374)

* Make some efficiency savings in pycbc_fit_sngls_over_multiparam

* Missing logging line in once case

* Add in progress reporting, in case of silent failure

* I thought we needed some leeway, but we don't

* TD comments

* Simplify comment

* Migrate to new gwdatafind API (#4383)

* Migrate to new gwdatafind API

* CC and one regression fix

* One more style change

* Don't need these options now

* Revert "Don't need these options now"

This reverts commit 6f2ed1b.

* Add a Fisher distribution for sky locations (#4209)

* Update transforms.py

Adding new_z_to_euler and rotate_euler from pylal.sphericalutils

* Update sky_location.py

Added new class FisherDist for drawing from sky locations (in ra, dec)

* Update sky_location.py

Included help message for fisherDist

* Update sky_location.py

* Code climate issues

* Code climate issues

* Code-climate changes

* Code climate issues

* Code climate issues

* Update transforms.py

changed lal.PI to numpy.pi in new_z_to_euler

* Update transforms.py

minor changes

* Update sky_location.py

updated docstring, did changes to make the code look more simple

* Update sky_location.py

typos corrected

* Update sky_location.py

code climate changes

* Update sky_location.py

* Update sky_location.py

Included the definitions rvs_polaz and rvs_radec

* Update transforms.py

Included the definitions decra2polaz and polaz2decra

* Update sky_location.py

* Update transforms.py

code climate issues fixed

* Update sky_location.py

code climate issues fixed

* Update sky_location.py

code climate issues

* Update sky_location.py

fixed code climate issues

* Update sky_location.py

fixed code climate issues

* Update sky_location.py

* Update pycbc/distributions/sky_location.py

Co-authored-by: Tito Dal Canton <[email protected]>

* Update sky_location.py

* Update sky_location.py

* Update transforms.py

modified polaz2decra to polaz2radec

* Update sky_location.py

* Update sky_location.py

Forgotten polaz2decra --> polaz2radec

* Update sky_location.py

Code climate fixes

* Update sky_location.py

Updated the code for checking the effectiveness of running from config file

* Create sky_location_old.py

* modified to include from config

* Modified to run from config file

* Added Fisher distribution to the lista of available distributions

* Delete sky_location_old.py

* Update sky_location.py

* Renamed Fisher to FisherSky

* Updated the name to FisherSky

* changed Fisher to FisherSky in __all__

* removed decra2polaz and polaz2radec to conversions

* Included def decra2polaz, polaz2radec in line 859

* Added angle_unit option

* removed decra2polaz and polaz2decra

* made carresction

* made corrections

* Corrected the typing error while removing recra2polaz

* Modified angle_unit and updated init file

* Empty line on 852 retrieved

* corrected

* Empty line on852 retrieved

* Fixed code climate issues

* Sigma conversion taken care of

* Fixed code climate issues

* Update sky_location.py

Codeclimate

* Update test_distributions.py

Adding fisher_sky to the list of distributions that do not undergo the distribution unittests.

* Update sky_location.py

Syntax fixed.

* Update sky_location.py complying to codeclimate

* Refactor using scipy's Rotation

* Improve docstring and make warning stricter

* Update pycbc/distributions/sky_location.py

Co-authored-by: Francesco Pannarale <[email protected]>

* Update pycbc/distributions/sky_location.py

right angle to correct angle

Co-authored-by: Francesco Pannarale <[email protected]>

* Update sky_location.py

logging issue solved

---------

Co-authored-by: Francesco Pannarale <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: P Prasia <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>

* add in several missing marginalization combinations (#4376)

* allow polarization marginalization with earth rotation

* add in missing marg combos

* fixes

* fixes

* fixes

* cleanup

* cc

* cc

* Try to fix CI tests (#4394)

* Switch FFTW/MKL order of preference

* Swap backend order and force gomp import

* FIx typo

* Do need openmpi in tox.ini

* Fix bug in MKL error function

* Force FFTW on this one.

* CC

* Requested Josh changes

* Add bank corner plot (#4339)

* Add corner plotting script for template banks

* Comment where premerger duration has been subtracted from template duration in bank_conversions

* add premerger_duration to bank_conversions.py

* Move premerger duration into bank_conversions

* Use bank's approximant

* Dont plot signular values as default

* CC complaints

* docstring

* TDC comments

* Keep approximant kwarg as default, TDC comments 2

* Added an OSError exception to MKL check (#4390)

* Add a possibility for defaultvalues sections in config files (#4385)

* Add a possibility for defaultvalues sections in config files

* do not allow [section] and [section-defaultvalues] options

* defaultvalues for top-level sections only

* underscore on add_ini_opts

* Remove default sections from get_subsections (#4399)

* Update setup-python action to v4 (#4400)

* Replace LOSC with GWOSC (#4401)

* Replace LOSC with GWOSC

* Fix leftover Python 2-ism

* Try to improve codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Try to fix codeclimate

* Exclude IFO sections when looping through the get_subsections (#4402)

* Exclude IFO sections when looping through the get_subsections for plots

* Add an example whihc would have previously failed into the search config

* Offline singles in workflow (#4386)

* Add single-detector options into workflow

* Fix a couple of isses which make results pages weird

* fix

* fix gen.sh to use correct workflow generator

* revert change made in error

* A couple of points which state coincident when it may not be

---------

Co-authored-by: Thomas Dent <[email protected]>
Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Gareth S Cabourn Davies <[email protected]>
Co-authored-by: Prasia-Pankunni <[email protected]>
Co-authored-by: Francesco Pannarale <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: P Prasia <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: Alex Nitz <[email protected]>
Co-authored-by: Arthur Tolley <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>

* Adding SEOBNRv5Peak key and function in the final frequency cutoff dictionary (#4408)

Co-authored-by: KHUN PHUKON <[email protected]>

* Injection kde (#4409)

* unexpected events

* added temp-volume in cl

* added injection kde

* significance bug for trigger_fit. A couple of minor fixes (#4407)

* Upgrades to pycbc_grb_inj_finder (#4364)

* retaining single ifo injection information

* Update pycbc_grb_inj_finder

Update according to requested changes from pull request review.

* Adding more specific comments

Co-authored-by: Francesco Pannarale <[email protected]>

* Adding more specific comments

out_dict changed to "output dictionary" in comment

Co-authored-by: Francesco Pannarale <[email protected]>

* Remove xlen

Removed xlen from cp_dataset_cont and instead calculated the length of xidentifier within the function.

* Reverting comment changes

* Remove extra comment

* Francesco's bug fix

Modified to simplify and correct where ifo information was jumbled together

* remove trig_dict and comment

* Removed extra lines

Removed line and datasets = {}

* Update to copyright comment and imports

Added our names and accounted for older python version.

---------

Co-authored-by: Francesco Pannarale <[email protected]>

* Simple bugfix (#4413)

Co-authored-by: Bhooshan Gadre <[email protected]>

* Move PyCBC venvs to new IGWN CVMFS server (#4412)

* First part of moving venv build to igwn

* TESTING HACK. REVERT ME BEFORE MERGING!!!

* Try this

* Testing

* Revert "Testing"

This reverts commit c0b9ea0.

* Add new CVMFS commands

* Remove rsync filter which isn't working

* Add error wrap abort handler

* Update tools/venv_transfer_commands.sh

Co-authored-by: Duncan Macleod <[email protected]>

* Fix Ian's idiocy (attempt to) and HostChecking

* Set to abort for one last test

* One final publish test

* Add a chmod on the root CVMFS directory

* Prepare for merging

* Fix mistake

---------

Co-authored-by: Duncan Macleod <[email protected]>

* Get FFT options in pycbc_condition_strain (#4404)

* pycbc_optimize_snr: Implement mass dependent bounds on spin parameters (#4176)

* Applying spin bounds based upon object mass~

* Updated reference to correct paper

* Adding spin bounds based on upper bounds of masses

* removing unnecessary additions

* Changing max mass2 to the right quantities

* Creating maxeta + reusing maxm1

* minor cleanup

* fixed min / max eta values

* fixed min / max eta

* Correcting maxm2 equation

* more readable/less redundant comment

---------

Co-authored-by: Thomas Dent <[email protected]>

* Allow SNR optimizer to use candidate point in initial array (#4393)

* allow SNR optimizer to use the candidate point in the initial array of test points

* update two LISA docs (#4417)

* Update injection_smbhb.ini

* Update lisa_smbhb_relbin.ini

* Update lisa_smbhb_relbin.ini

* Update lisa_smbhb_relbin.ini

* Update lisa_smbhb_ldc_pe.rst

* Allow pycbc_multi_inspiral to search over a sky grid (#4380)

* Allow pycbc_multi_inspiral to search over a sky grid

* Adding the possibility to give either a sky grid or a sky position

* Change the names of the command lines --ra and --dec to --latitude and --longitude to match the changes made in pycbc_multi_inspiral and remove --processing-scheme because it makes the example crash

* Compute the antenna pattern only once for each sky position by keeping fp and fc together

* Replace the number of detectors from a hardcoded value to a variable

* add TDI AE PSD with confusion noise for LISA (#4389)

* add TDI AE PSD with confusion noise

* Update analytical_space.py

* fix cc issue

* Update analytical.py

* EM brute (#4299)

* Space before colons in eos_utils.py help messages

* Enable extrapolation in eos_utils.py when NS is out of bounds

* Same check on shape in primary_mass and secondary mass

* Function to swap masses and spins of object 1 and object2

* Allow user to request for remnant_mass calculations: 1) extrapolation, 2) swapping object 1 and object 2 if mass2 > mass1, 3) treat as a (dark) BBH and NSBH with NS mass greater than a value probided by the user

* Renamned swap_companions to ensure_obj1_is_primary and generalized it

* codeclimate

* codeclimate

* Fixed line that was too long

* Fixed docstring and using keywords explicitly in all remnant_mass_* calls

* Renamed max_ns_mass as ns_bh_mass_boundary and improved help messages of functions

* Improved docstings

* Comprehensions in pycbc_brute_bank

* Complying to naming convention for spin components in spherical coordinates

* Update conversions.py

Fixing single code climate complaint.

* Prevent duplicates of the HLV page_ifar plot on the summary page (#4421)

* Prevent duplicates of the HLV page_ifar plot on the summary page

* Rename 'coincidences' as appropriate

* add high frequency sky location dependent response for long detectors (#4377)

* reset commit, add in high freqeuncy response to detector

* update

* fixes

* Make pegasus 5.0.5 work (#4276)

* Revert "try pinning pegasus version (#4265)"

This reverts commit 398e135.

* Set for_planning flag for subworkflow output map

* Add version check so we don't break 5.0.3

* Also include 5.0.4

* make ci faster (#4414)

* update

* move long log into debug level, allow to set debug level

* fix workflow

* more explicit control of running external scripts for docs setup at same time

* fixes

* update

* try this

* try this ...

* update

* fixes

* update

* updates

* mistake in example search config file (#4425)

* mistake in example search config file

* add failure if search runs too long

* Range_temp kde (#4424)

* unexpected events

* added temp-volume in cl

* setting the range for temp_kde

* setting min based on max kde

* logging for min template kde ratio

---------

Co-authored-by: Thomas Dent <[email protected]>

* Pygrb offline workflow (#4288)

* Fixed indexing and autochisq argument in pycbc_multi_inspiral

* Only Hanford is labelled with a 1: avoiding pycbc_pygrb_plot_snr_timeseries failure.  trigs rather than trigs_or_injs as variable name in pycbc_pygrb_plot_coh_ifosnr

* Simplified and update pycbc_pygrb_plot_injs_results; adapted pycbc_pygrb_pp_workflow

* Forgotten comma

* Massive commit that stitches together pycbc_make_offline_grb_workflow and pycbc_pygrb_pp_workflow, by picking up the latter as a SubWorkflow.

* Massive commit that stitches together pycbc_make_offline_grb_workflow and pycbc_pygrb_pp_workflow, by picking up the latter as a SubWorkflow.

* Code-climate fixes

* More code-climate

* Import error flagged by code-climate

* Forgotten import fixes

* Minor cleanup of comments or commented out code lines

* Syntax fix

* Defining 2 empty file lists

* First part of PR 4288 review comments

* One more TODO

* opt_to_file is now configparser_value_to_file in pycbc/workflow/core.py

* Upgraded configparser_value_to_file with attrs keyword argument

* Using configparser_value_to_file throughout the PyGRB workflow generator

* Removing 2 unused PyGRB functions

* Removed LigolwCBCAlignTotalSpinExecutable, LigolwCBCJitterSkylocExecutable, and PycbcDarkVsBrightInjectionsExecutable support in general and removed LalappsInspinjExecutable support from PyGRB

* Codeclimate

* bank_veto_file variable needs to be a FileList, not a File

* Bug with tags for single IFO SNR plots workflow generator

* Fixing pycbc_pygrb_plot_coh_ifosnr

* Fixing layout of chi-square plots and section label for individual detector SNRs

* Codeclimate

* Codeclimate

* Codeclimate

* Removed FIXME for testing purposes

* First changes following up on PR review

* Second round of changes following up on PR review

* Flagging with an underscore grb utility functions not used outside the files they are defined in

* Flagging with an underscore grb utility functions not used outside the files they are defined in

* Removed _get_antenna_factors from PyGRB utily file

* Removed commented out function from pycbc_pygrb_plot_injs_results

* Substituted old unused code with a TODO

* Adding injection set name as an option in pycbc_pygrb_efficiency

* extend --> append for single files

* Documenting setup_pygrb_pp_workflow more appropriately

* Reverted name of _get_antenna_responses to get_antenna_responses; cleaner handling of bank-veto-bank-file for pycbc_multi_inspiral job setup.

* Line shortened

* Avoid skymap v1.1.0 (#4429)

* only download LISA response file once (#4423)

* Update analytical_space.py

* Update analytical_space.py

* Introduced a new arguement --skymap-only-ifos in pycbc_live (#4346)

* Introduced a new arguement --skymap-only-ifos in pycbc_live

* Default skymap_only_ifos changed

* skymap_only_ifos as an attribute of LiveEventManager

* Test skymaps with V1 as skymap_only_ifos

* singles with skymap_only_ifos

* Removing commented lines etc

* A default seed for SNR opt

* Poking CI tests

I remove an unnecessary FIXME in the comments, but mostly doing this to restart the CI tests, which seem to have not linked up right.

---------

Co-authored-by: Souradeep Pal <[email protected]>
Co-authored-by: Ian Harry <[email protected]>

* Fix stageout (#4428)

* Fix the help message for stageout script

* Fix the help message for stageout and copy map scripts

---------

Co-authored-by: connor-mcisaac <[email protected]>

* more physical rwrap for LISA SMBHB/SOBHB injection (follow-up fix 2) (#4350)

* more physical rwrap

* fix cc issues

* Update waveform.py

* Update companion.txt

* Update requirements.txt

* Update requirements.txt

* Update companion.txt

* 10 times tau

* Update companion.txt

* Update requirements.txt

* Update setup.py

* add check

* Update waveform.py

* allow user to choose rwarp

* fix cc issue

* set 0.2 as minimum rwrap

* Update waveform.py

* Remove most X509/globus links and hooks from PyCBC (#4375)

* Remove most X509/globus links and hooks from PyCBC

* Need to update this one as well

* Don't need these options now

* Update docs/workflow/pycbc_make_coinc_search_workflow.rst

Co-authored-by: Gareth S Cabourn Davies <[email protected]>

---------

Co-authored-by: Gareth S Cabourn Davies <[email protected]>

* fftw_fix (#4418)

* Allow pycbc inspiral to run with only numpy fft

* Remove Unneeded function from npfft.py

---------

Co-authored-by: Neeresh Kumar Perla <[email protected]>

* Add argument verification for singles (#4365)

* Add argument verification for singles

* Apply suggestions from code review

* remove --enable-single-detector-background

* Some more option-checking, and making fixed ifar option actually possible!

* fix options in example

* CC, typo

* CC

* Move versioning page generation into its own executable (#4431)

* Move versioning page generation into its own executable

* Allow duplicated executables

* 🤦

* CC changes

* CC

* save_fig_with_metadata cannot be imported from versioning

* Don't like spaces in arguments

* Ian's comments

* Dont pass executables as files

* Same type output from get_code_version_numbers

* Rename --longitude and --latitude back to --ra and --dec (#4433)

* bugfix - allow non-running of singles (#4439)

* bugfix - allow non-running of singles

* CC

* Revrt wrongly-added test chnages

* Add IMRPhenomXAS template duration (#4410)

* Add IMRPhenomXAS template duration

* removing assertion

* fixed fomatting for codeclimate

* Adding PhenomXAS duration and removing space fixes

* removing extra new line

---------

Co-authored-by: Bhooshan Gadre <[email protected]>

* Treat CVMFS better (#4440)

* dqsegdb2 update (#4442)

* dqsegdb2 update

* Update requirements-igwn.txt

Co-authored-by: Duncan Macleod <[email protected]>

---------

Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Duncan Macleod <[email protected]>

---------

Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: Gareth S Cabourn Davies <[email protected]>
Co-authored-by: Arthur Tolley <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: Praveen Kumar <[email protected]>
Co-authored-by: Prasia-Pankunni <[email protected]>
Co-authored-by: Francesco Pannarale <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: P Prasia <[email protected]>
Co-authored-by: Tito Dal Canton <[email protected]>
Co-authored-by: Alex Nitz <[email protected]>
Co-authored-by: Khun Sang Phukon <[email protected]>
Co-authored-by: KHUN PHUKON <[email protected]>
Co-authored-by: ETVincent <[email protected]>
Co-authored-by: Bhooshan Uday Varsha Gadre <[email protected]>
Co-authored-by: Bhooshan Gadre <[email protected]>
Co-authored-by: Duncan Macleod <[email protected]>
Co-authored-by: Shichao Wu <[email protected]>
Co-authored-by: hoangstephanie <[email protected]>
Co-authored-by: SouradeepPal <[email protected]>
Co-authored-by: Souradeep Pal <[email protected]>
Co-authored-by: Ian Harry <[email protected]>
Co-authored-by: connor-mcisaac <[email protected]>
Co-authored-by: Neeresh Kumar <[email protected]>
Co-authored-by: Neeresh Kumar Perla <[email protected]>
  • Loading branch information
26 people authored Jul 26, 2023
1 parent 9454725 commit 7296778
Show file tree
Hide file tree
Showing 139 changed files with 3,539 additions and 2,549 deletions.
12 changes: 11 additions & 1 deletion .github/workflows/basic-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
matrix:
os: [ubuntu-20.04]
python-version: [3.7, 3.8, 3.9, '3.10', '3.11']
test-type: [help, unittest, search, inference, docs]
test-type: [unittest, search, docs]
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
Expand All @@ -39,6 +39,16 @@ jobs:
run: |
export LAL_DATA_PATH=$PWD
tox -e py-${{matrix.test-type}}
- name: check help messages work
if: matrix.test-type == 'unittest'
run: |
export LAL_DATA_PATH=$PWD
tox -e py-help
- name: run inference tests
if: matrix.test-type == 'search'
run: |
export LAL_DATA_PATH=$PWD
tox -e py-inference
- name: store documentation page
if: matrix.test-type == 'docs' && matrix.python-version == '3.8'
uses: actions/upload-artifact@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build_venv.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
uses: actions/checkout@v1
-
name: "Set up Python"
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.9
-
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docker-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
uses: actions/checkout@v1
-
name: "Set up Python"
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.9
-
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/inference-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
steps:
- uses: actions/checkout@v1
- name: Set up Python
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.8
- name: install condor
Expand All @@ -25,7 +25,7 @@ jobs:
wget -qO - https://download.pegasus.isi.edu/pegasus/gpg.txt | sudo apt-key add -
echo "deb https://download.pegasus.isi.edu/pegasus/ubuntu bionic main" | sudo tee -a /etc/apt/sources.list
sudo apt-get -o Acquire::Retries=3 update
sudo apt-get -o Acquire::Retries=3 install pegasus=5.0.3-1+ubuntu18
sudo apt-get -o Acquire::Retries=3 install pegasus
- run: sudo apt-get -o Acquire::Retries=3 install *fftw3* intel-mkl*
- name: Install pycbc
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/mac-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
steps:
- uses: actions/checkout@v1
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- run: |
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/search-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@ concurrency:
jobs:
build:
runs-on: ubuntu-20.04
timeout-minutes: 90
steps:
- uses: actions/checkout@v1
- name: Set up Python
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.8
- name: install condor
Expand All @@ -29,7 +30,7 @@ jobs:
wget -qO - https://download.pegasus.isi.edu/pegasus/gpg.txt | sudo apt-key add -
echo "deb https://download.pegasus.isi.edu/pegasus/ubuntu bionic main" | sudo tee -a /etc/apt/sources.list
sudo apt-get -o Acquire::Retries=3 update
sudo apt-get -o Acquire::Retries=3 install pegasus=5.0.3-1+ubuntu18
sudo apt-get -o Acquire::Retries=3 install pegasus
- run: sudo apt-get -o Acquire::Retries=3 install *fftw3* intel-mkl*
- name: Install pycbc
run: |
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/tmpltbank-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
steps:
- uses: actions/checkout@v1
- name: Set up Python
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.8
- name: install condor
Expand All @@ -29,7 +29,7 @@ jobs:
wget -qO - https://download.pegasus.isi.edu/pegasus/gpg.txt | sudo apt-key add -
echo "deb https://download.pegasus.isi.edu/pegasus/ubuntu bionic main" | sudo tee -a /etc/apt/sources.list
sudo apt-get -o Acquire::Retries=3 update
sudo apt-get -o Acquire::Retries=3 install pegasus=5.0.3-1+ubuntu18
sudo apt-get -o Acquire::Retries=3 install pegasus
- run: sudo apt-get -o Acquire::Retries=3 install *fftw3* intel-mkl*
- name: Install pycbc
run: |
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/workflow-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
steps:
- uses: actions/checkout@v1
- name: Set up Python
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.8
- name: install condor
Expand All @@ -34,7 +34,7 @@ jobs:
wget -qO - https://download.pegasus.isi.edu/pegasus/gpg.txt | sudo apt-key add -
echo "deb https://download.pegasus.isi.edu/pegasus/ubuntu bionic main" | sudo tee -a /etc/apt/sources.list
sudo apt-get -o Acquire::Retries=3 update
sudo apt-get -o Acquire::Retries=3 install pegasus=5.0.3-1+ubuntu18
sudo apt-get -o Acquire::Retries=3 install pegasus
- run: sudo apt-get -o Acquire::Retries=3 install *fftw3* intel-mkl*
- name: Install pycbc
run: |
Expand Down
136 changes: 122 additions & 14 deletions bin/all_sky_search/pycbc_template_kde_calc
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,27 @@ parser.add_argument('--signal-file', help='File with parameters of GW signals '
'for KDE calculation')
parser.add_argument('--template-file', required=True, help='Hdf5 file with '
'template masses and spins')
parser.add_argument('--injection-file', help='Hdf5 file with masses and spins')
parser.add_argument('--min-mass', type=float, default=None,
help='Used only on signal masses: remove all'
'signal events with mass2 < min_mass')
parser.add_argument('--min-snr', type=float, default=None,
help='Used only in injections case: remove all'
'injection events < min_snr')
parser.add_argument('--nfold-signal', type=int,
help='Number of k-folds for signal KDE cross validation')
parser.add_argument('--nfold-template', type=int,
help='Number of k-folds for template KDE cross validation')
parser.add_argument('--nfold-injection', type=int,
help='Number of k-folds for injection KDE cross validation')
parser.add_argument('--fit-param', nargs='+', required=True,
help='Parameters over which KDE is calculated')
parser.add_argument('--log-param', nargs='+', choices=['True', 'False'],
required=True)
parser.add_argument('--output-file', required=True, help='Name of .hdf output')
parser.add_argument('--make-signal-kde', action='store_true')
parser.add_argument('--make-template-kde', action='store_true')
parser.add_argument('--make-injection-kde', action='store_true')
parser.add_argument('--fom-plot', help='Make a FOM plot for cross-validation'
' and save it as this file')
parser.add_argument('--alpha-grid', type=float, nargs="+",
Expand All @@ -49,17 +56,23 @@ parser.add_argument('--bw-grid', type=float, nargs='+',
help='Grid of choices of global bandwidth')
parser.add_argument('--extra-cpt-fraction', type=float,
help='Fraction of the extra component in the signal density')
parser.add_argument('--temp-volume', type=float,
parser.add_argument('--temp-volume', type=float,
help='Volume covered by the template bank')
parser.add_argument('--seed', type=int,
help='Random number generator seed')
parser.add_argument('--mchirp-downsample-power', type=float,
help='Exponent value for the power law distribution')
parser.add_argument('--min-ratio', type=float,
help='Minimum ratio for template_kde relative to the maximum')
parser.add_argument('--verbose', action='store_true')
args = parser.parse_args()
init_logging(verbose=args.verbose, format='%(asctime)s %(message)s')


assert len(args.fit_param) == len(args.log_param)
if args.make_signal_kde and args.make_template_kde:
parser.error("Choose only one option out of --make-signal-kde and \
--make-template-kde")
if args.make_signal_kde + args.make_template_kde + args.make_injection_kde != 1:
parser.error("Choose exactly one option out of --make-signal-kde, \
--make-template-kde, or --make-injection-kde")


if (args.extra_cpt_fraction is None and args.temp_volume is not None) or \
Expand Down Expand Up @@ -133,7 +146,7 @@ def optimizedparam(sampleval, bwgrid, alphagrid, nfold=2):

# Obtaining template parameters
temp_file = h5py.File(args.template_file, 'r')
mass1 = temp_file['mass1']
mass1 = temp_file['mass1'][:]
tid = numpy.arange(len(mass1)) # Array of template ids
mass_spin = triggers.get_mass_spin(temp_file, tid)

Expand Down Expand Up @@ -162,23 +175,66 @@ temp_samples = numpy.vstack((template_pars)).T


if args.make_template_kde:
logging.info('Starting optimization of template KDE parameters')
optbw, optalpha = optimizedparam(temp_samples, alphagrid=args.alpha_grid,
bwgrid=args.bw_grid, nfold=args.nfold_template)
logging.info('Bandwidth %.4f, alpha %.2f' % (optbw, optalpha))
logging.info('Evaluating template KDE')
template_kde = kde_awkde(temp_samples, temp_samples, alp=optalpha,
gl_bandwidth=optbw)

# Rejection sampling to reduce computational load
if args.mchirp_downsample_power is not None:
logging.info('Downsampling with mchirp power '
f'{args.mchirp_downsample_power}')
f_dest.attrs['mchirp_downsample_power'] = args.mchirp_downsample_power
try:
mchirp_index = args.fit_param.index('mchirp')
except:
raise ValueError("mchirp does not exist in args.fit_param")

mc_vals = template_pars[mchirp_index]
if args.log_param[mchirp_index]:
mc_vals = numpy.exp(mc_vals)
power_vals = mc_vals ** args.mchirp_downsample_power
probabilities = power_vals / numpy.max(power_vals)
if args.seed is not None:
f_dest.attrs['seed'] = args.seed
numpy.random.seed(args.seed)
rand_nums = numpy.random.uniform(0, 1, len(mass1))
ind = rand_nums < probabilities
logging.info(f'{ind.sum()} templates after downsampling')
kde_train_samples = temp_samples[ind]
f_dest.create_dataset("kde_train_samples", data=kde_train_samples)

logging.info('Starting optimization of template KDE parameters')
optbw, optalpha = optimizedparam(kde_train_samples, alphagrid=args.alpha_grid,
bwgrid=args.bw_grid, nfold=args.nfold_template)
logging.info('Bandwidth %.4f, alpha %.2f' % (optbw, optalpha))
logging.info('Evaluating template KDE')
template_kde = kde_awkde(kde_train_samples, temp_samples, alp=optalpha,
gl_bandwidth=optbw)

# Compensation factor for downsampling of templates
template_kde *= 1. / probabilities
if args.min_ratio is not None:
logging.info(f'Applying minimum template KDE ratio {args.min_ratio}')
f_dest.attrs['min-kde-ratio'] = args.min_ratio
min_val = args.min_ratio * numpy.max(template_kde)
template_kde = numpy.maximum(template_kde, min_val)

else:
logging.info('Starting optimization of template KDE parameters')
optbw, optalpha = optimizedparam(temp_samples, alphagrid=args.alpha_grid,
bwgrid=args.bw_grid, nfold=args.nfold_template)
logging.info('Bandwidth %.4f, alpha %.2f' % (optbw, optalpha))
logging.info('Evaluating template KDE')
template_kde = kde_awkde(temp_samples, temp_samples, alp=optalpha,
gl_bandwidth=optbw)

f_dest.create_dataset("data_kde", data=template_kde)
f_dest.attrs['stat'] = "template-kde_file"
f_dest.attrs['template-file'] = args.template_file
f_dest.attrs['template-file'] = args.template_file


def signal_kde_extra_cpt(signal_kde, frac=None, volume=None):
# Calculate the constant density value for the additional component
const_density = 1 / volume
# Calculate the modified signal density
modified_kde = (frac * signal_kde) + ((1 - frac) * const_density)
modified_kde = (1 - frac) * signal_kde + frac * const_density
return modified_kde


Expand Down Expand Up @@ -213,6 +269,7 @@ if args.make_signal_kde:
or 'False'")

signal_samples = numpy.vstack((signal_pars)).T
f_dest.create_dataset("kde_train_samples", data=signal_samples)
logging.info('Starting optimization of signal KDE parameters')
optbw, optalpha = optimizedparam(signal_samples, bwgrid=args.bw_grid,
alphagrid=args.alpha_grid, nfold=args.nfold_signal)
Expand All @@ -221,13 +278,64 @@ if args.make_signal_kde:
signal_kde = kde_awkde(signal_samples, temp_samples,
alp=optalpha, gl_bandwidth=optbw)
if args.extra_cpt_fraction is not None and args.temp_volume is not None:
f_dest.attrs.update({'extra-fraction-used': args.extra_cpt_fraction,
'volume': args.temp_volume})
modified_kde = signal_kde_extra_cpt(signal_kde, frac=args.extra_cpt_fraction,
volume=args.temp_volume)
else:
modified_kde = signal_kde
f_dest.create_dataset("data_kde", data=modified_kde)
f_dest.attrs['stat'] = "signal-kde_file"


if args.make_injection_kde:
inj_pars = []
inj_file = h5py.File(args.injection_file, 'r')
f_dest.attrs['injection-file'] = args.injection_file
snr = inj_file["events"]["snr_net"][:]
N_original = len(snr)
if args.min_snr:
idx = snr > args.min_snr
snr = snr[idx]
logging.info('%i triggers out of %i with SNR > %s' %
(len(snr), N_original, str(args.min_snr)))
else:
idx = numpy.full(N_original, True)
mass1 = inj_file["events"]["mass1_detector"][idx]
mass2 = inj_file["events"]["mass2_detector"][idx]
spin1z = inj_file["events"]["spin1z"][idx]
spin2z = inj_file["events"]["spin2z"][idx]

for param, slog in zip(args.fit_param, args.log_param):
pvals = triggers.get_param(param, args, mass1, mass2, spin1z, spin2z)
if slog in ['False']:
logging.info('Using param: %s', param)
inj_pars.append(pvals)
elif slog in ['True']:
logging.info('Using log param: %s', param)
inj_pars.append(numpy.log(pvals))
else:
raise ValueError("invalid log param argument, use 'True', or 'False'")

inj_samples = numpy.vstack((inj_pars)).T
f_dest.create_dataset("kde_train_samples", data=inj_samples)
logging.info('Starting optimization of injection KDE parameters')
optbw, optalpha = optimizedparam(inj_samples, alphagrid=args.alpha_grid,
bwgrid=args.bw_grid, nfold=args.nfold_injection)
logging.info('Bandwidth %.4f, alpha %.2f' % (optbw, optalpha))
logging.info('Evaluating injection KDE')
injection_kde = kde_awkde(inj_samples, temp_samples, alp=optalpha,
gl_bandwidth=optbw)
if args.extra_cpt_fraction is not None and args.temp_volume is not None:
f_dest.attrs.update({'extra-fraction-used': args.extra_cpt_fraction,
'volume': args.temp_volume})
modified_kde = signal_kde_extra_cpt(injection_kde, frac=args.extra_cpt_fraction,
volume=args.temp_volume)
else:
modified_kde = injection_kde
f_dest.create_dataset("data_kde", data=modified_kde)
f_dest.attrs['stat'] = "injection-kde_file"

f_dest.attrs['alpha'] = optalpha
f_dest.attrs['bandwidth'] = optbw
f_dest.close()
Expand Down
Loading

0 comments on commit 7296778

Please sign in to comment.