diff --git a/README.md b/README.md index adc2b44d..3bb249f6 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,17 @@ for the full installation instructions. We also offer [Docker](https://i4ds.github.io/Karabo-Pipeline/container.html) images. + +Quick Look with no Installation +------------------------------- + +If you are curious to see whether Karabo is for you or if you want to try it out before you install something, then this is for you: we offer a demo installation on Renkulab. This demo was created for a workshop at Swiss SKA Days in September 2024. It has been kept up to date ever since. + +The demo installation can be found as [SwissSKADays-Karabo-Workshop](https://renkulab.io/projects/menkalinan56/swissskadays-karabo-workshop). You can start a server (free of cost) and start using the Karabo pipeline without an account. However, if you want to save your work, you need to log in using your GitHub account, your ORCID id, or your edu-ID. You then fork the project. Changes you make will be saved to your GitLab repository linked to your Renkulab accout. + +A good starting point may be the slide deck of the workshop. You can find it in the folder `documents`. The code in the slides is available as Jupyter notebooks in the folder `notebooks`. Those help you get started. + + Contribute to Karabo --------------------- We are very happy to accept contributions, either as code or even just bug reports! When writing code, diff --git a/doc/src/conf.py b/doc/src/conf.py index 5bd76f5f..3bd3b2b1 100644 --- a/doc/src/conf.py +++ b/doc/src/conf.py @@ -25,7 +25,7 @@ project = "Karabo-Pipeline" copyright = "2024, i4ds" -author = "i4ds, ETH" +author = "i4ds, ETHZ" # The full version, including alpha/beta/rc tags release = __version__ diff --git a/doc/src/examples/example_structure.md b/doc/src/examples/example_structure.md index 169d6206..61bf47d9 100644 --- a/doc/src/examples/example_structure.md +++ b/doc/src/examples/example_structure.md @@ -8,6 +8,12 @@ The following example showcases a simple telescope simulation using the main pip ``` +## Show telescope config + +```python + +``` + ## Imaging The notebook [imaging.ipynb](https://github.com/i4Ds/Karabo-Pipeline/blob/main/karabo/examples/imaging.ipynb), shows how to use different dirty imaging and image cleaning algorithms. @@ -44,11 +50,7 @@ See the script [line_emission.py](https://github.com/i4Ds/Karabo-Pipeline/blob/m This simulation begins with a `SkyModel` instance, and with the definition of the desired `Observation` and `Telescope` details. Then, the `InterferometerSimulation` instance uses the requested backend (OSKAR and RASCIL are currently supported) to compute the corresponding visibilities, and the desired `DirtyImager` instance is used to convert the visibilities into dirty images. Optionally, we can include primary beam effects and correct for such effects in the final dirty images. Finally, we can mosaic different dirty images into one larger image using the `ImageMosaicker` class. -## Show telescope config -```python - -``` ![Image](../images/telescope.png) @@ -60,3 +62,13 @@ The name of the directory is given the name of the telescope followed by configu More details can be find in the OSKAR documentation and source code: https://ska-telescope.gitlab.io/sim/oskar/telescope_model/telescope_model.html +## Using our Demo Installation +We offer an always up-to-date demo installation for Karabo on Renkulab. This demo was created for a workshop at Swiss SKA Days in September 2024. It can be found here: + +[SwissSKADays-Karabo-Workshop](https://renkulab.io/projects/menkalinan56/swissskadays-karabo-workshop) + +You do not need an account to use the demo. Just hit the green 'Start' button in the top right corner. However, if you want to save your work you need to log in using your GitHub account, your ORCID id, or your edu-ID. Then fork the project. Now changes you make will be saved to your GitLab repository linked to your Renkulab accout. + +The free server on Renkulab has limited resources. You will not be able to run much larger simulations than those provided in the demo notebooks. + +A good starting point may be the slide deck of the workshop. You can find it in the folder documents. The code in the slides is available as Jupyter notebooks in the folder notebooks. Those help you get started. \ No newline at end of file diff --git a/doc/src/examples/examples.md b/doc/src/examples/examples.md index 81ec1f57..974d2ad1 100644 --- a/doc/src/examples/examples.md +++ b/doc/src/examples/examples.md @@ -44,6 +44,15 @@ observation = Observation( simulation.run_simulation(telescope, sky, observation) ``` +## Show telescope config + +```python +from karabo.simulation.telescope import Telescope + +telescope = Telescope.constructor("EXAMPLE") +telescope.plot_telescope(file="example_telescope.png") +``` + ## Imaging The notebook [imaging.ipynb](https://github.com/i4Ds/Karabo-Pipeline/blob/main/karabo/examples/imaging.ipynb), shows how to use different dirty imaging and image cleaning algorithms. @@ -80,14 +89,7 @@ See the script [line_emission.py](https://github.com/i4Ds/Karabo-Pipeline/blob/m This simulation begins with a `SkyModel` instance, and with the definition of the desired `Observation` and `Telescope` details. Then, the `InterferometerSimulation` instance uses the requested backend (OSKAR and RASCIL are currently supported) to compute the corresponding visibilities, and the desired `DirtyImager` instance is used to convert the visibilities into dirty images. Optionally, we can include primary beam effects and correct for such effects in the final dirty images. Finally, we can mosaic different dirty images into one larger image using the `ImageMosaicker` class. -## Show telescope config - -```python -from karabo.simulation.telescope import Telescope -telescope = Telescope.constructor("EXAMPLE") -telescope.plot_telescope(file="example_telescope.png") -``` ![Image](../images/telescope.png) @@ -99,3 +101,13 @@ The name of the directory is given the name of the telescope followed by configu More details can be find in the OSKAR documentation and source code: https://ska-telescope.gitlab.io/sim/oskar/telescope_model/telescope_model.html +## Using our Demo Installation +We offer an always up-to-date demo installation for Karabo on Renkulab. This demo was created for a workshop at Swiss SKA Days in September 2024. It can be found here: + +[SwissSKADays-Karabo-Workshop](https://renkulab.io/projects/menkalinan56/swissskadays-karabo-workshop) + +You do not need an account to use the demo. Just hit the green 'Start' button in the top right corner. However, if you want to save your work you need to log in using your GitHub account, your ORCID id, or your edu-ID. Then fork the project. Now changes you make will be saved to your GitLab repository linked to your Renkulab accout. + +The free server on Renkulab has limited resources. You will not be able to run much larger simulations than those provided in the demo notebooks. + +A good starting point may be the slide deck of the workshop. You can find it in the folder documents. The code in the slides is available as Jupyter notebooks in the folder notebooks. Those help you get started. diff --git a/doc/src/images/telescope.png b/doc/src/images/telescope.png index f61a7dfc..3aa834b8 100644 Binary files a/doc/src/images/telescope.png and b/doc/src/images/telescope.png differ diff --git a/doc/src/index.rst b/doc/src/index.rst index 9d24e079..efa0d4f3 100644 --- a/doc/src/index.rst +++ b/doc/src/index.rst @@ -16,8 +16,9 @@ Karabo is a starting point for the `Square Kilometer Array Image: """Creates a clean image from a dirty image or from visibilities. - Args: + Arguments: visibility: Visibility from which a clean image should be created. dirty_fits_path: Path to dirty image FITS file that should be reused to create a clean image. If None, dirty image will be diff --git a/karabo/imaging/imager_rascil.py b/karabo/imaging/imager_rascil.py index 4aed7a24..8d4cf26c 100644 --- a/karabo/imaging/imager_rascil.py +++ b/karabo/imaging/imager_rascil.py @@ -155,66 +155,65 @@ class RascilImageCleanerConfig(ImageCleanerConfig): Adds parameters specific to RascilImageCleaner. - Attributes: - imaging_npixel (int): see ImageCleanerConfig - imaging_cellsize (float): see ImageCleanerConfig - ingest_dd (List[int]): Data descriptors in MS to read (all must have the same - number of channels). Defaults to [0]. - ingest_vis_nchan (Optional[int]): Number of channels in a single data - descriptor in the MS. Defaults to None. - ingest_chan_per_vis (int): Number of channels per blockvis (before any average). - Defaults to 1. - imaging_nchan (int): Number of channels per image. Defaults to 1. - imaging_w_stacking (Union[bool, str]): Use the improved w stacking method - in Nifty Gridder?. Defaults to True. - imaging_flat_sky (Union[bool, str]): If using a primary beam, normalise to - flat sky? Defaults to False. - override_cellsize (bool): Override the cellsize if it is above - the critical cellsize? Defaults to False. - imaging_uvmax (Optional[float]): TODO. Defaults to None. - imaging_uvmin (float): TODO. Defaults to 0. - imaging_dft_kernel (Optional[DftKernelType]): DFT kernel: cpu_looped | gpu_raw. - Defaults to None. - client (Optional[Client]): Dask client. Defaults to None. - use_dask (bool): Use dask? Defaults to False. - n_threads (int): n_threads per worker. Defaults to 1. - use_cuda (bool): Use CUDA for Nifty Gridder? Defaults to False. - img_context (ImageContextType): Which nifty gridder to use. Defaults to "ng". - clean_algorithm (CleanAlgorithmType): Deconvolution algorithm - (hogbom or msclean or mmclean). Defaults to "hogbom". - clean_beam (Optional[Dict[CleanBeamInDegType, float]]): major axis, minor axis, - position angle (deg). Defaults to None. - clean_scales (List[int]): Scales for multiscale clean (pixels) e.g. [0, 6, 10]. - Defaults to [0]. - clean_nmoment (int): Number of frequency moments in mmclean - (1=constant, 2=linear). Defaults to 4. - clean_nmajor (int): Number of major cycles in cip or ical. Defaults to 5. - clean_niter (int): Number of minor cycles in CLEAN. Defaults to 1000. - clean_psf_support (int): Half-width of psf used in cleaning (pixels). - Defaults to 256. - clean_gain (float): Clean loop gain. Defaults to 0.1. - clean_threshold (float): Clean stopping threshold (Jy/beam). Defaults to 1e-4. - clean_component_threshold (Optional[float]): Sources with absolute flux - > this level (Jy) are fit or extracted using skycomponents. - Defaults to None. - clean_component_method (CleanComponentMethodType): Method to convert sources - in image to skycomponents: "fit" in frequency or "extract" actual values. - Defaults to "fit". - clean_fractional_threshold (float): Fractional stopping threshold for major - cycle. Defaults to 0.3. - clean_facets (int) Number of overlapping facets in faceted clean along each - axis. Defaults to 1. - clean_overlap (int): Overlap of facets in clean (pixels). Defaults to 32. - clean_taper (CleanTaperType): Type of interpolation between facets in - deconvolution: none or linear or tukey. Defaults to "tukey". - clean_restore_facets (int): Number of overlapping facets in restore step - along each axis. Defaults to 1. - clean_restore_overlap (int): Overlap of facets in restore step (pixels). - Defaults to 32. - clean_restore_taper (CleanTaperType): Type of interpolation between facets in - restore step (none, linear or tukey). Defaults to "tukey". - clean_restored_output (CleanRestoredOutputType): Type of restored image output: - taylor, list, or integrated. Defaults to "list". + :param imaging_npixel: see ImageCleanerConfig + :param imaging_cellsize: see ImageCleanerConfig + :param ingest_dd: Data descriptors in MS to read (all must have the same \ + number of channels). Defaults to [0]. + :param ingest_vis_nchan: Number of channels in a single data \ + descriptor in the MS. Defaults to None. + :param ingest_chan_per_vis: Number of channels per blockvis (before any average). \ + Defaults to 1. + :param imaging_nchan: Number of channels per image. Defaults to 1. + :param imaging_w_stacking: Use the improved w stacking method \ + in Nifty Gridder. Defaults to True. + :param imaging_flat_sky: If using a primary beam, normalise to \ + flat sky. Defaults to False. + :param override_cellsize: Override the cellsize if it is above \ + the critical cellsize. Defaults to False. + :param imaging_uvmax: TODO. Defaults to None. + :param imaging_uvmin: TODO. Defaults to 0. + :param imaging_dft_kernel: DFT kernel: cpu_looped | gpu_raw.\ + Defaults to None. + :param client: Dask client. Defaults to None. + :param use_dask: Use dask? Defaults to False. + :param n_threads: n_threads per worker. Defaults to 1. + :param use_cuda: Use CUDA for Nifty Gridder? Defaults to False. + :param img_context: Which nifty gridder to use. Defaults to "ng". + :param clean_algorithm: Deconvolution algorithm \ + (hogbom or msclean or mmclean). Defaults to "hogbom". + :param clean_beam: major axis, minor axis,\ + position angle (deg). Defaults to None. + :param clean_scales: Scales for multiscale clean (pixels) e.g. [0, 6, 10].\ + Defaults to [0]. + :param clean_nmoment: Number of frequency moments in mmclean\ + (1=constant, 2=linear). Defaults to 4. + :param clean_nmajor: Number of major cycles in cip or ical. Defaults to 5. + :param clean_niter: Number of minor cycles in CLEAN. Defaults to 1000. + :param clean_psf_support: Half-width of psf used in cleaning (pixels).\ + Defaults to 256. + :param clean_gain: Clean loop gain. Defaults to 0.1. + :param clean_threshold: Clean stopping threshold (Jy/beam). Defaults to 1e-4. + :param clean_component_threshold: Sources with absolute flux \ + > this level (Jy) are fit or extracted using skycomponents.\ + Defaults to None. + :param clean_component_method: Method to convert sources \ + in image to skycomponents: "fit" in frequency or "extract" actual values. \ + Defaults to "fit". + :param clean_fractional_threshold: Fractional stopping threshold for major \ + cycle. Defaults to 0.3. + :param clean_facets: Number of overlapping facets in faceted clean along each\ + axis. Defaults to 1. + :param clean_overlap: Overlap of facets in clean (pixels). Defaults to 32. + :param clean_taper: Type of interpolation between facets in \ + deconvolution: none or linear or tukey. Defaults to "tukey". + :param clean_restore_facets: Number of overlapping facets in restore step \ + along each axis. Defaults to 1. + :param clean_restore_overlap: Overlap of facets in restore step (pixels). \ + Defaults to 32. + :param clean_restore_taper: Type of interpolation between facets in \ + restore step (none, linear or tukey). Defaults to "tukey". + :param clean_restored_output: Type of restored image output:\ + taylor, list, or integrated. Defaults to "list". """ ingest_dd: List[int] = field(default_factory=_create_ingest_dd_default_value) diff --git a/karabo/simulation/beam.py b/karabo/simulation/beam.py index a4cf6954..f62c72c7 100644 --- a/karabo/simulation/beam.py +++ b/karabo/simulation/beam.py @@ -144,10 +144,11 @@ def make_cst_from_arr( arr: ArrayLike, output_file_path: str, ) -> None: - """ - Takes array of dimensions (*,8), and returns a cst files + """Takes array of dimensions (* ,8), and returns a cst files :param arr: - :return: cst file with given output filename + :param out_file_path: The name of the output file. Must include path. + + :return: cst file with given output filename """ line1 = ( "Theta [deg.] Phi [deg.] Abs(Dir.) Horiz(Abs) " @@ -175,9 +176,11 @@ def get_meerkat_uhfbeam( sampling_step: int = 80, ) -> Tuple[NDArray[np.float_], NDArray[np.float_], NDArray[np.float_]]: """ - - :param pol: - :param beamextent: + :param f: The frequency for which the beam is simulated (MHz) + :param pol: The polarisation of the beam. Valid values are "H", "V", and "I" + :param beamextentx: + :param beamextendy: + :param sampling_step: :return: """ beam = JimBeam("MKAT-AA-UHF-JIM-2020") diff --git a/karabo/simulation/interferometer.py b/karabo/simulation/interferometer.py index 7d1c6e0b..fe2480d0 100644 --- a/karabo/simulation/interferometer.py +++ b/karabo/simulation/interferometer.py @@ -87,96 +87,95 @@ class FilterUnits(enum.Enum): class InterferometerSimulation: """ Class containing all configuration for the Interferometer Simulation. + :ivar channel_bandwidth_hz: The channel width, in Hz, used to simulate bandwidth - smearing. (Note that this can be different to the - frequency increment if channels do not cover a - contiguous frequency range.) + smearing. (Note that this can be different to the + frequency increment if channels do not cover a + contiguous frequency range). :ivar time_average_sec: The correlator time-average duration, in seconds, used - to simulate time averaging smearing. + to simulate time averaging smearing. :ivar max_time_per_samples: The maximum number of time samples held in memory - before being written to disk. + before being written to disk. :ivar correlation_type: The type of correlations to produce. Any value of Enum - CorrelationType + CorrelationType :ivar uv_filter_min: The minimum value of the baseline UV length allowed by the - filter. Values outside this range are not evaluated + filter. Values outside this range are not evaluated :ivar uv_filter_max: The maximum value of the baseline UV length allowed by the - filter. Values outside this range are not evaluated. + filter. Values outside this range are not evaluated. :ivar uv_filter_units: The units of the baseline UV length filter values. - Any value of Enum FilterUnits + Any value of Enum FilterUnits :ivar force_polarised_ms: If True, always write the Measurement Set in polarised - format even if the simulation was run in the single - polarisation 'Scalar' (or Stokes-I) mode. If False, - the size of the polarisation dimension in the - Measurement Set will be determined by the simulation - mode. + format even if the simulation was run in the single + polarisation 'Scalar' (or Stokes-I) mode. If False, + the size of the polarisation dimension in the + Measurement Set will be determined by the simulation mode. :ivar ignore_w_components: If enabled, baseline W-coordinate component values will - be set to 0. This will disable W-smearing. Use only if - you know what you're doing! + be set to 0. This will disable W-smearing. Use only if + you know what you're doing! :ivar noise_enable: If true, noise is added. :ivar noise_seed: Random number generator seed. :ivar noise_start_freq: The start frequency in Hz for which noise is included, if - noise is set to true. + noise is set to true. :ivar noise_inc_freq: The frequency increment in Hz, if noise is set to true. :ivar noise_number_freq: The number of frequency taken into account, if noise is set - to true. + to true. :ivar noise_rms_start: Station RMS (noise) flux density range start value, in Jy. - The range is expanded linearly over the number of frequencies - for which noise is defined. + The range is expanded linearly over the number of frequencies + for which noise is defined. :ivar noise_rms_end: Station RMS (noise) flux density range end value, in Jy. The - range is expanded linearly over the number of frequencies for - which noise is defined. + range is expanded linearly over the number of frequencies for + which noise is defined. :ivar noise_rms: The specifications for the RMS noise value: - Telescope model: values are loaded from files in the telescope - model directory. - Data file: values are loaded from the specified file. - Range: values are evaluated according to the specified range - parameters (Default). - The noise values are specified in Jy and represent the RMS noise of - an unpolarised source in terms of flux measured in a single - polarisation of the detector. + Telescope model: values are loaded from files in the telescope + model directory. + Data file: values are loaded from the specified file. + Range: values are evaluated according to the specified range + parameters (Default). + The noise values are specified in Jy and represent the RMS noise of + an unpolarised source in terms of flux measured in a single + polarisation of the detector. :ivar noise_freq: The list of frequencies for which noise values are defined: - Telescope model: frequencies are loaded from a data file in - the telescope model directory. - Observation settings: frequencies are defined by the observation - settings. - Data file: frequencies are loaded from the specified data file. - Range: frequencies are specified by the range parameters - (Default). + Telescope model: frequencies are loaded from a data file in + the telescope model directory. + Observation settings: frequencies are defined by the observation + settings. + Data file: frequencies are loaded from the specified data file. + Range: frequencies are specified by the range parameters (Default). :ivar enable_array_beam: If true, then the contribution to the station beam from - the array pattern (given by beam-forming the antennas in - the station) is evaluated. + the array pattern (given by beam-forming the antennas in + the station) is evaluated. :ivar enable_numerical_beam: If true, make use of any available numerical element - pattern files. If numerical pattern data are missing, - the functional type will be used instead. - :ivar beam_polX: currently only considered for `ObservationLong` - :ivar beam_polX: currently only considered for `ObservationLong` + pattern files. If numerical pattern data are missing, + the functional type will be used instead. + :ivar beam_polX: currently only considered for 'ObservationLong' + :ivar beam_polX: currently only considered for 'ObservationLong' :ivar use_gpus: Set to true if you want to use gpus for the simulation :ivar client: The dask client to use for the simulation :ivar split_idxs_per_group: The indices of the sky model to split for each group - of workers. If None, the sky model will not be split. - Useful if the sky model is too large to fit into the - memory of a single worker. Group index should be - strictly monotonic increasing. + of workers. If None, the sky model will not be split. + Useful if the sky model is too large to fit into the + memory of a single worker. Group index should be + strictly monotonic increasing. :ivar precision: For the arithmetic use you can choose between "single" or - "double" precision + "double" precision :ivar station_type: Here you can choose the type of each station in the - interferometer. You can either disable all station beam - effects by choosing "Isotropic beam". Or select one of the - following beam types: - "Gaussian beam", "Aperture array" or "VLA (PBCOR)" + interferometer. You can either disable all station beam + effects by choosing "Isotropic beam". Or select one of the + following beam types: + "Gaussian beam", "Aperture array" or "VLA (PBCOR)" :ivar enable_power_pattern: If true, gauss_beam_fwhm_deg will be taken in as - power pattern. + power pattern. :ivar gauss_beam_fwhm_deg: If you choose "Gaussian beam" as station type you need - specify the full-width half maximum value at the - reference frequency of the Gaussian beam here. - Units = degrees. If enable_power_pattern is True, - gauss_beam_fwhm_deg is in power pattern, otherwise - it is in field pattern. + specify the full-width half maximum value at the + reference frequency of the Gaussian beam here. + Units = degrees. If enable_power_pattern is True, + gauss_beam_fwhm_deg is in power pattern, otherwise + it is in field pattern. :ivar gauss_ref_freq_hz: The reference frequency of the specified FWHM, in Hz. :ivar ionosphere_fits_path: The path to a fits file containing an ionospheric screen - generated with ARatmospy. The file parameters - (times/frequencies) should coincide with the planned - observation. + generated with ARatmospy. The file parameters + (times/frequencies) should coincide with the planned observation. + """ def __init__( diff --git a/karabo/simulation/sky_model.py b/karabo/simulation/sky_model.py index 80f2552a..247e92d5 100644 --- a/karabo/simulation/sky_model.py +++ b/karabo/simulation/sky_model.py @@ -495,37 +495,37 @@ class SkyModel: Class containing all information of the to be observed Sky. `SkyModel.sources` is a `xarray.DataArray` - ( https://docs.xarray.dev/en/stable/generated/xarray.DataArray.html ). + (https://docs.xarray.dev/en/stable/generated/xarray.DataArray.html). `np.ndarray` are also supported as input type for `SkyModel.sources`, - however, the values in `SkyModel.sources` are converted to `xarray.DataArray`. + however, the values in `SkyModel.sources are converted to `xarray.DataArray`. `SkyModel.compute` method is used to load the data into memory as a numpy array. It should be called after all the filtering and other operations are completed and to avoid doing the same calculation multiple them when e.g. on a cluster. - :ivar sources: List of all point sources in the sky as `xarray.DataArray`. - The source_ids reside in `SkyModel.source_ids` if provided - through `xarray.sources.coords` with an arbitrary string key - as index or `np.ndarray` as idx SOURCES_COLS. - A single point source is described using the following col-order: - - - [0] right ascension (deg) - - [1] declination (deg) - - [2] stokes I Flux (Jy) - - [3] stokes Q Flux (Jy): defaults to 0 - - [4] stokes U Flux (Jy): defaults to 0 - - [5] stokes V Flux (Jy): defaults to 0 - - [6] reference_frequency (Hz): defaults to 0 - - [7] spectral index (N/A): defaults to 0 - - [8] rotation measure (rad / m^2): defaults to 0 - - [9] major axis FWHM (arcsec): defaults to 0 - - [10] minor axis FWHM (arcsec): defaults to 0 - - [11] position angle (deg): defaults to 0 - - [12] true redshift: defaults to 0 - - [13] observed redshift: defaults to 0 - - [14] object-id: just for `np.ndarray` - it is removed in the `xr.DataArray` - and exists then in `xr.DataArray.coords` as index. + :ivar sources: List of all point sources in the sky as `xarray.DataArray`. + The source_ids reside in `SkyModel.source_ids` if provided + through `xarray.sources.coords` with an arbitrary string key + as index or `np.ndarray` as idx SOURCES_COLS. + A single point source is described using the following col-order: + + - [0] right ascension (deg) + - [1] declination (deg) + - [2] stokes I Flux (Jy) + - [3] stokes Q Flux (Jy): defaults to 0 + - [4] stokes U Flux (Jy): defaults to 0 + - [5] stokes V Flux (Jy): defaults to 0 + - [6] reference_frequency (Hz): defaults to 0 + - [7] spectral index (N/A): defaults to 0 + - [8] rotation measure (rad / m^2): defaults to 0 + - [9] major axis FWHM (arcsec): defaults to 0 + - [10] minor axis FWHM (arcsec): defaults to 0 + - [11] position angle (deg): defaults to 0 + - [12] true redshift: defaults to 0 + - [13] observed redshift: defaults to 0 + - [14] object-id: just for `np.ndarray` + it is removed in the `xr.DataArray` + and exists then in `xr.DataArray.coords` as index. :ivar wcs: World Coordinate System (WCS) object representing the coordinate transformation between pixel coordinates and celestial coordinates (e.g., right ascension and declination). @@ -740,12 +740,12 @@ def add_point_sources(self, sources: _SkySourcesType) -> None: """Add new point sources to the sky model. :param sources: `np.ndarray` with shape (number of sources, 1 + SOURCES_COLS), - where you can place the "source_id" at index SOURCES_COLS. - OR an `xarray.DataArray` with shape (number of sources, SOURCES_COLS), - where you can place the "source_id" at `xarray.DataArray.coord` - or use `SkyModel.source_ids` later. + where you can place the "source_id" at index SOURCES_COLS. + OR an `xarray.DataArray` with shape (number of sources, SOURCES_COLS), + where you can place the "source_id" at `xarray.DataArray.coord` + or use `SkyModel.source_ids` later. - The column indices correspond to: + The column indices correspond to: - [0] right ascension (deg) - [1] declination (deg) @@ -762,6 +762,7 @@ def add_point_sources(self, sources: _SkySourcesType) -> None: - [12] true redshift: defaults to 0 - [13] observed redshift: defaults to 0 - [14] source id (object): is in `SkyModel.source_ids` if provided + """ if (len(sources.shape) == 2) and (sources.shape[0] == 0): warn( @@ -908,8 +909,9 @@ def filter_by_radius( :param ra0_deg: Phase center right ascension :param dec0_deg: Phase center declination :param indices: Optional parameter, if set to True, - we also return the indices of the filtered sky copy - :return sky: Filtered copy of the sky + we also return the indices of the filtered sky copy + :return: Filtered copy of the sky + :rtype: SkyModel """ copied_sky = SkyModel.copy_sky(self) if copied_sky.sources is None: @@ -1166,7 +1168,7 @@ def explore_sky( :param phase_center: [RA,DEC] :param stokes: `SkyModel` stoke flux :param idx_to_plot: If you want to plot only a subset of the sources, set - the indices here. + the indices here. :param xlim: RA-limit of plot :param ylim: DEC-limit of plot :param figsize: figsize as tuple @@ -1178,13 +1180,13 @@ def explore_sky( :param cbar_label: color bar label :param with_labels: Plots object ID's if set? :param wcs: If you want to use a custom astropy.wcs, ignores `phase_center` if - set + set :param wcs_enabled: Use wcs transformation? :param filename: Set to path/fname to save figure (set extension to fname to - overwrite .png default) + overwrite .png default) :param block: Whether or not plotting should block the rest of the program :param kwargs: matplotlib kwargs for scatter & Collections, e.g. customize `s`, - `vmin` or `vmax` + `vmin` or `vmax` """ # To avoid having to read the data multiple times, we read it once here if self.sources is None: @@ -1308,7 +1310,7 @@ def read_healpix_file_to_sky_model_array( :param file: hdf5 file path (healpix format) :param channel: Channels of observation (between 0 and maximum numbers of - channels of observation) + channels of observation) :param polarisation: 0 = Stokes I, 1 = Stokes Q, 2 = Stokes U, 3 = Stokes V :return: """ @@ -1344,8 +1346,7 @@ def sources(self, value: Optional[_SkySourcesType]) -> None: have different dtypes (issue 3004). Just set "# type: ignore [assignment]" in case you don't have exactly an `xarray`. - Args: - value: sources, `xarray.DataArray` or `np.ndarray` + :param value: sources, `xarray.DataArray` or `np.ndarray` """ self._sources = None self._sources_dim_sources = XARRAY_DIM_0_DEFAULT @@ -1825,27 +1826,34 @@ def get_sky_model_from_fits( chunks: Optional[_ChunksType] = "auto", memmap: bool = False, ) -> _TSkyModel: - """Creates a sky-model from `fits_file`. The following formats are supported: - - Each data-array of the .fits file maps to a single `SkyModel.sources` column. + """ + Creates a sky-model from `fits_file`. The following formats are supported: + + - Each data-array of the .fits file maps to a single `SkyModel.sources` \ + column. + - Frequency of some columns is encoded in col-names of the .fits file. - Args: - fits_file: The .fits file to create the sky-model from. - prefix_mapping: Formattable col-names of .fits file. If `encoded_freq` is - not None, the freq-encoded field-values must have '{0}' as placeholder. - unit_mapping: Mapping from col-unit to `astropy.unit`. - units_sources: Units of `SkyModel.sources`. - min_freq: Filter by min-freq in Hz? May increase file-reading significantly. - max_freq: Filter by max-freq in Hz? May increase file-reading significantly. - encoded_freq: Unit of col-name encoded frequency, if the .fits file has - it's ref-frequency encoded in the col-names. - chunks: Coerce the array's data into dask arrays with the given chunks. - Might be useful for reading larger-than-memory .fits files. - memmap: Whether to use memory mapping when opening the FITS file. - Allows for reading of larger-than-memory files. + :param fits_file: The .fits file to create the sky-model from. + :param prefix_mapping: Formattable col-names of .fits file. If `encoded_freq` + is not None, the freq-encoded field-values must have '{0}' + as placeholder. + :param unit_mapping: Mapping from col-unit to `astropy.unit`. + :param units_sources: Units of `SkyModel.sources`. + :param min_freq: Filter by min-freq in Hz. May slow down reading of + file significantly. + :param max_freq: Filter by max-freq in Hz. May slow down reading of + file significantly. + :param encoded_freq: Unit of col-name encoded frequency, if the .fits file has + it's ref-frequency encoded in the col-names. + :param chunks: Coerce the array's data into dask arrays with the given chunks. + Might be useful for reading larger-than-memory .fits files. + :param memmap: Whether to use memory mapping when opening the FITS file. + Allows for reading of larger-than-memory files. + + :return: Sky model with according sources from `fits_file`. + :rtype: SkyModel - Returns: - sky-model with according sources from `fits_file`. """ def chunks_fun(arr: xr.DataArray) -> xr.DataArray: @@ -1953,30 +1961,28 @@ def get_GLEAM_Sky( @classmethod def get_sample_simulated_catalog(cls: Type[_TSkyModel]) -> _TSkyModel: - """ - Downloads a sample simulated HI source catalog and generates a sky + """ Downloads a sample simulated HI source catalog and generates a sky model using the downloaded data. The catalog size is around 8MB. - Source: - The simulated catalog data was provided by Luis Machado + Source: The simulated catalog data was provided by Luis Machado (https://github.com/lmachadopolettivalle) in collaboration with the ETHZ Cosmology Research Group. - Returns: - SkyModel: The corresponding sky model. + :return: The corresponding sky model. The sky model contains the following information: - - - 'Right Ascension' (ra): The right ascension coordinates - of the celestial objects. - - 'Declination' (dec): The declination coordinates of the - celestial objects. + - 'Right Ascension' (ra): The right ascension coordinates \ + of the celestial objects. + - 'Declination' (dec): The declination coordinates of the \ + celestial objects. - 'Flux' (i): The flux measurements of the celestial objects. - - 'Observed Redshift': Additional observed redshift information - of the celestial objects. + + - 'Observed Redshift': Additional observed redshift information \ + of the celestial objects. Note: Other properties such as 'stokes_q', 'stokes_u', 'stokes_v', - 'ref_freq', 'spectral_index', 'rm', 'major', 'minor', 'pa', and 'id' + 'ref_freq', 'spectral_index', 'rm', 'major', 'minor', 'pa', and 'id' are not included in the sky model. + :rtype: SkyModel """ survey = HISourcesSmallCatalogDownloadObject() @@ -2168,21 +2174,23 @@ def convert_to_backend( ) -> Union[SkyModel, List[SkyComponent]]: """Convert an existing SkyModel instance into a format acceptable by a desired backend. - backend: Determines how to return the SkyModel source catalog. + + :param backend: Determines how to return the SkyModel source catalog. OSKAR: return the current SkyModel instance, since methods in Karabo support OSKAR-formatted source np.array values. RASCIL: convert the current source array into a list of RASCIL SkyComponent instances. - desired_frequencies_hz: List of frequencies corresponding to start - of desired frequency channels. This field is required - to convert sources into RASCIL SkyComponents. + :param desired_frequencies_hz: List of frequencies corresponding to start + of desired frequency channels. This field is required + to convert sources into RASCIL SkyComponents. The array contains starting frequencies for the desired channels. E.g. [100e6, 110e6] corresponds to 2 frequency channels, which start at 100 MHz and 110 MHz, both with a bandwidth of 10 MHz. - channel_bandwidth_hz: Used if desired_frequencies_hz has only one element. + channel_bandwidth_hz: Used if desired_frequencies_hz has only one element. Otherwise, bandwidth is determined as the delta between the first two entries in desired_frequencies_hz. - verbose: Determines whether to display additional print statements. + :param verbose: Determines whether to display additional print statements. + """ if backend is SimulatorBackend.OSKAR: diff --git a/karabo/simulation/telescope.py b/karabo/simulation/telescope.py index 8307005c..7e284412 100644 --- a/karabo/simulation/telescope.py +++ b/karabo/simulation/telescope.py @@ -263,12 +263,13 @@ def constructor( :param backend: Underlying package to be used for the telescope configuration, since each package stores the arrays in a different format. Defaults to OSKAR. - :raises: ValueError if the combination of input parameters is invalid. + :raise ValueError: If the combination of input parameters is invalid. Specifically, if the requested telescope requires a version, but an invalid version (or no version) is provided, or if the requested telescope name is not supported by the requested backend. - :returns: Telescope instance. + + :return: Telescope instance """ if backend is SimulatorBackend.OSKAR: # Explicitly cast name depending on whether it requires a telescope version @@ -340,21 +341,22 @@ def constructor( @classmethod def __convert_to_karabo_telescope(cls, instr_name: str) -> Telescope: - """Converts a site saved in RASCIl data format into a Karabo Telescope. - This function acts as an adapter to make the functionality in Telescope - class work for a RASCIL telescope. Namely the functions max_baseline() - and get_baseline_lengths(). - It derives the necessary data structures from the RASCIL_configuration - and fits them into those of the Telescope class. The resuting class is - a SimulatorBackend.RASCIL but has the stations: List[Station] - list filled as well. Nevertheless, it should only be used as a RASCIL - telescope class. + """ + Converts a site saved in RASCIl data format into a Karabo Telescope. + This function acts as an adapter to make the functionality in Telescope + class work for a RASCIL telescope. Namely the functions max_baseline() + and get_baseline_lengths(). + It derives the necessary data structures from the RASCIL_configuration + and fits them into those of the Telescope class. The resuting class is + a SimulatorBackend.RASCIL but has the stations: List[Station] + list filled as well. Nevertheless, it should only be used as a RASCIL + telescope class. :param instr_name: The name of the instrument to convert. - + :raise ValueError: If instr_name is not a valid RASCIL telescope :returns: An instance of Karabo Telescope. :rtype: karabo.simulation.telescope.Telecope - :raises: ValueError if instr_name is not a valid RASCIL telescope + """ config = create_named_configuration(instr_name) @@ -473,10 +475,11 @@ def add_antenna_to_station( :param horizontal_y: north coordinate relative to the station center in metres :param horizontal_z: altitude of antenna :param horizontal_x_coordinate_error: east coordinate error - relative to the station center in metres + relative to the station center in metres :param horizontal_y_coordinate_error: north coordinate error - relative to the station center in metres + relative to the station center in metres :param horizontal_z_coordinate_error: altitude of antenna error + :return: """ if station_index >= 0 and station_index < len(self.stations): @@ -622,7 +625,7 @@ def read_OSKAR_tm_file(cls, path: DirPathType) -> Telescope: """Reads an OSKAR telescope model from disk and returns an object of karabo.simulation.telescope.Telescope - :param path: Path to a valid telescope model (extemsion *.tm) + :param path: Path to a valid telescope model (extemsion * .tm) :return: A karabo.simulation.telescope.Telescope object. Importantn: The object has the backend set to SimulatorBackend.OSKAR. :raises: A karabo.error.KaraboError if the path does not exit, @@ -800,18 +803,18 @@ def create_baseline_cut_telescope( tel: Telescope, tm_path: Optional[DirPathType] = None, ) -> Tuple[DirPathType, Dict[str, str]]: - """Returns a telescope model for telescope `tel` with baseline lengths - only between `lcut` and `hcut` metres. - - Args: - lcut: Lower cut - hcut: Higher cut - tel: Telescope to cut off - tm_path: .tm file-path to save the cut-telescope. - `tm_path` will get overwritten if it already exists. - - Returns: - .tm file-path & station-name conversion (e.g. station055 -> station009) + """ + Returns a telescope model for telescope `tel` with baseline lengths + only between `lcut` and `hcut` metres. + + :param lcut: Lower cut + :param hcut: Higher cut + :param tel: Telescope to cut off + :param tm_path: .tm file-path to save the cut-telescope. + `tm_path` will get overwritten if it already exists. + + :return: .tm file-path & station-name + conversion (e.g. station055 -> station009) """ if tel.path is None: raise KaraboError( diff --git a/karabo/sourcedetection/evaluation.py b/karabo/sourcedetection/evaluation.py index f6c9e35e..f1a3d55c 100644 --- a/karabo/sourcedetection/evaluation.py +++ b/karabo/sourcedetection/evaluation.py @@ -29,14 +29,17 @@ def __init__( source_detection: ISourceDetectionResult, ) -> None: """Class that holds the mapping of a source detection to truth mapping. + :param sky: `SkyModel` where the `assignment` comes from :param ground_truth: 2xn array of pixel positions of ground truth :param assignments: jx3 np.ndarray where each row represents an assignment: - - first column is the `ground_truth` index - - second column is the predicted `source_detection.detected_sources` index - - third column is the euclidean distance between the assignment + + - first column is the `ground_truth` index + - second column is the predicted `source_detection.detected_sources` index + - third column is the euclidean distance between the assignment :param sky_idxs: Sky sources indices of `SkyModel` from `assignment` :param source_detection: SourceDetectionResult from a previous source-detection + """ self.sky = sky self.ground_truth = ground_truth @@ -109,37 +112,38 @@ def automatic_assignment_of_ground_truth_and_prediction( top_k: int = 3, ) -> NDArray[np.float_]: """Automatic assignment of the predicted sources `predicted` to the - ground truth `gtruth`. The strategy is the following (similar to - `AUTOMATIC SOURCE DETECTION IN ASTRONOMICAL IMAGES, P.61, - Marc MASIAS MOYSET, 2014`): - Each distance between the predicted and the ground truth sources is calculated. - Any distances > `max_dist` are not considered. - Assign the closest distance from the predicted and ground truth. - Repeat the assignment, until every source from the gtruth has an - assignment if possible, not allowing any double assignments from the predicted - sources to the ground truth and vice versa. So each ground truth source - should be assigned with a predicted source if at least one was in range - and the predicted source assigned to another ground truth source before. - If there are duplicate sources (e.g. same source, different frequency), the - duplicate sources are removed and the assignment is done on the remaining. + ground truth `gtruth`. The strategy is the following (similar to + `AUTOMATIC SOURCE DETECTION IN ASTRONOMICAL IMAGES, P.61, + Marc MASIAS MOYSET, 2014`): + Each distance between the predicted and the ground truth sources is + calculated. Any distances > `max_dist` are not considered. + Assign the closest distance from the predicted and ground truth. + Repeat the assignment, until every source from the gtruth has an + assignment if possible, not allowing any double assignments from the + predicted sources to the ground truth and vice versa. So each ground truth + source should be assigned with a predicted source if at least one was + in range and the predicted source assigned to another ground truth source + before. If there are duplicate sources (e.g. same source, different + frequency), the duplicate sources are removed and the assignment is done + on the remaining. :param ground_truth: nx2 np.ndarray with the ground truth pixel - coordinates of the catalog + coordinates of the catalog :param detected: kx2 np.ndarray with the predicted pixel - coordinates of the image + coordinates of the image :param max_dist: maximal allowed euclidean distance for assignment - (in pixel domain) + (in pixel domain) :param top_k: number of top predictions to be considered in scipy.spatial. - KDTree. A small value could lead to imperfect results. + KDTree. A small value could lead to imperfect results. :return: nx3 np.ndarray where each row represents an assignment - - first column represents the ground truth index - (return is sorted by this column) a minus index means a ground-truth - source with no allocated prediction - - second column represents the predicted index - a minus index means a predicted source with no allocated ground-truth - - third column represents the euclidean distance between the assignment - a "inf" means no allocation between ground-truth and prediction - of that source + - first column represents the ground truth index + (return is sorted by this column) a minus index means a ground-truth + source with no allocated prediction + - second column represents the predicted index + a minus index means a predicted source with no allocated ground-truth + - third column represents the euclidean distance between the assignment + a "inf" means no allocation between ground-truth and prediction + of that source """ # Check if there are duplicate sources and if yes, remove them diff --git a/karabo/sourcedetection/result.py b/karabo/sourcedetection/result.py index e1a9acd1..5e70a419 100644 --- a/karabo/sourcedetection/result.py +++ b/karabo/sourcedetection/result.py @@ -117,7 +117,7 @@ def detect_sources_in_image( Parameters ---------- - cls : Type[_SourceDetectionResultType] + cls : Type[SourceDetectionResultType] The class on which this method is called. image : Image or List[Image] Image object for source detection. Can be a single image or a list of @@ -132,12 +132,12 @@ def detect_sources_in_image( greater than 1 requires Dask. overlap : int, default 0 The overlap between split parts of the image in pixels. - **kwargs : Any + ** kwargs : Any Additional keyword arguments to pass to PyBDSF.process_image function. Returns ------- - Optional[List[_SourceDetectionResultType]] + Optional[List[SourceDetectionResultType]] A list of detected sources, or None if all pixels in the image are blanked or on failure.