Bug fix

• Fix for h5py>3.0.
• Improved docs re automatic gridding.

Automatic gridding

The simulation class comes new with an automatic gridding functionality, which should make it much easier to compute CSEM data. With that the entire optimization routine was improved too. See the API docs for more info of the relevant implementation.

• simulation:

  • Simulation: New gridding options 'single', 'frequency', 'source', and 'both'; new default is 'single'.
  • compute() takes a new argument, min_offset. If observed=True, it will add Gaussian random noise according to the standard deviation of the data; it will set receivers responses below the minimum offset to NaN.
  • There is no longer a reference model.
  • misfit and gradient can now handle observations with NaN's.

• survey: A Survey has new attributes standard_error, noise_floor, and relative_error.

• optimize: Completely changed misfit and data-weighting to more sensible functions.

• cli:

  • As a consequence of the changes the data_weight_opts got removed.
  • New sections [data] to select the wanted data and [gridding_opts] for options of the automatic gridding.
  • Section [simulation] has a new parameter min_offset (for creating observed data).
  • Output has a new parameter n_observations if misfit or gradient were called, which is the number of observations that were used to compute the misfit.

• meshes:

  • New functions construct_mesh, get_origin_widths, good_mg_cell_nr and other, smaller helper routines.
  • Deprecated the old meshing routines get_hx_h0, get_cell_numbers, get_stretched_h, get_domain, get_hx; they will be removed in the future.
  • Default of good_mg_cell_nr changed, and the documentation (and verbosity) with regards to «good» number of cells was improved.

• Bug fixes:

  • maps: Fixed the mapping of the gradients (Conductivity is the only mapping that was not affected by this bug).

• Removed deprecated features:

  • models.Model: Removed parameters res_{x;y;z}.
  • io.save: Removed deprecated parameter backend.
  • io.save: Removed default, file extension has to be provided.

CLI

• New Module cli for command-line interaction:

  The command-line interface can currently be used to forward model an entire Simulation, and also to compute the misfit of it with respect to some data and the gradient of the misfit function. See the section "CLI interface" in the documentation for more info.

Note that, while cli (v0.13.0) and optimize (v0.12.0) are implemented, they are still in development and are likely going to change throughout the next two minor releases or so.

• Other changes:

  • solver: Changes in verbosity for emg3d.solve:

    • New default verbosity is 1 (only warnings; before it was 2).
    • Verbosities {-1;0;1} remain unchanged.
    • Verbosities {2;3;4} => {3;4;5}.
    • New verbosity 2: Only shows a one-liner at the end (plus warnings).

  • survey and simulation: to_file and from_file have new a parameter name, to store and load with a particular name instead of the default survey/simulation (useful when storing, e.g., many surveys in one file).

  • survey: stores new also the reference-data; different data (observed, reference) is contained in a data-dict when storing.

  • simulation: takes new a verb parameter.

  • optimize:

    • Gradient now possible for arbitrarily rotated sources and receivers.
    • Falls back to synthetic instead of observed now if reference not found.

  • io: np.bool_ are converted back to bool when loading.

  • Re-arrange, improve, and update documentation.

Survey & Simulation

This is a big release with many new features, and unfortunately not completely
backwards compatible. The main new features are the new Survey and
Simulation classes, as well as some initial work for optimization
(misfit, gradient). Also, a Model can now be a resistivity model, a
conductivity model, or the logarithm (natural or base 10) therefore. Receivers
can now be arbitrarily rotated, just as the sources. In addition to the
existing soft-dependencies empymod, discretize, and h5py there are
the new soft-dependencies xarray and tqm; discretize is now much tighter
integrated. For the new survey and simulation classes xarray is a required
dependency. However, the only hard dependency remain scipy and numba, if
you use emg3d purely as a solver. Data reading and writing has new a
JSON-backend, in addition to the existing HDF5 and NumPy-backends.

In more detail:

• Modules:

  • surveys (new; requires xarray):

    • Class surveys.Survey, which combines sources, receivers, and data.
    • Class surveys.Dipole, which defines electric or magnetic point dipoles
      and finite length dipoles.

  • simulations (new; requires xarray; soft-dependency tqdm):

    • Class simulations.Simulation, which combines a survey with a model. A
      simulation computes the e-field (and h-field) asynchronously using
      concurrent.futures. This class will include automatic, source- and
      frequency-dependent gridding in the future. If tqdm is installed it
      displays a progress bar for the asynchronous computation. Note that the
      simulation class has still some limitations, consult the class
      documentation.

  • models:

    • Model instances take new the parameters property_{x;y;z} instead of
      res_{x;y;z}. The properties can be either resistivity, conductivity, or
      log_{e;10} thereof. What is actually provided has to be defined with the
      parameter mapping. By default, it remains resistivity, as it was until
      now. The keywords res_{x;y;z} are deprecated, but still accepted at
      the moment. The attributes model.res_{x;y;z} are still available too,
      but equally deprecated. However, it is no longer possible to
      assign values to these attributes, which is a backwards
      incompatible change.
    • A model knows now how to interpolate itself from its grid to another grid
      (interpolate2grid).

  • maps:

    • New mappings for models.Model instances: The mappings take care of
      how to transform the investigation variable to conductivity and back, and
      how it affects its derivative.
    • New interpolation routine edges2cellaverages.

  • fields:

    • Function get_receiver_response (new), which returns the response
      for arbitrarily rotated receivers.

    • Improvements to Field and SourceField:

      • _sval and _smu0 not stored any longer, derived from _freq.
      • SourceField is now using the copy() and from_dict() from its
        parents class Field.

  • io:

    • File-format json (new), writes to a hierarchical, plain json file.
    • Deprecated the use of backend, it uses the file extension of
      fname instead.
    • This means .npz (instead of numpy), .h5 (instead of h5py), and
      new .json.
    • New parameter collect_classes, which can be used to switch-on
      collection of the main classes in root-level dictionaries. By default,
      they are no longer collected (changed).

  • meshes:

    • meshes.TensorMesh new inherits from discretize if installed.
    • Added __eq__ to models.TensorMesh to compare meshes.

  • optimize (new)

    • Functionalities related to inversion (data misfit, gradient, data
      weighting, and depth weighting). This module is in an early stage, and
      the API will likely change in the future. Current functions are misfit,
      gradient (using the adjoint-state method), and data_weighting. These
      functionalities are best accessed through the Simulation class.

• Dependencies:

  • empymod is now a soft dependency (no longer a hard dependency), only
    required for utils.Fourier (time-domain modelling).
  • Existing soft dependency discretize is now baked straight into meshes.
  • New soft dependency xarray for the Survey class (and therefore also for
    the Simulation class and the optimize module).
  • New soft dependency tqdm for nice progress bars in asynchronous
    computation.

• Deprecations and removals:

  • Removed deprecated functions data_write and data_read.
  • Removed all deprecated functions from utils.

• Miscellaneous:

  • Re-organise API-docs.
  • Much bookkeeping (improve error raising and checking; chaining errors,
    numpy types, etc).

Refactor

Grand refactor with new internal layout. Mainly splitting-up utils into
smaller bits. Most functionalities (old names) are currently retained in
utils and it should be mostly backwards compatible for now, but they are
deprecated and will eventually be removed. Some previously deprecated functions
were removed, however.

• Removed deprecated functions:

  • emg3d.solver.solver (use emg3d.solver.solve instead).
  • Aliases of emg3d.io.data_write and emg3d.io.data_read in emg3d.utils.

• Changes:

  • SourceField has now the same signature as Field (this might break your
    code if you called SourceField directly, with positional arguments, and
    not through get_source_field).

  • More functions and classes in the top namespace.

  • Replaced core.l2norm with scipy.linalg.norm, as SciPy 1.4 got the
    following PR: scipy/scipy#10397 (reason to raise
    minimum SciPy to 1.4).

  • Increased minimum required versions of dependencies to

    • scipy>=1.4.0 (raised from 1.1, see note above)
    • empymod>=2.0.0 (no min requirement before)
    • numba>=0.45.0 (raised from 0.40)

• New layout

  • njitted -> core.
  • utils split in fields, meshes, models, maps, and utils.

• Bugfixes:

  • Fixed to_dict, from_dict, and copy for the SourceField.
  • Fixed io for SourceField, that was not implemented properly.

Zero Source

• Bug fixes:

  • Checks now if provided source-field is zero, and exists gracefully if so, returning a zero electric field. Until now it failed with a division-by-zero error.

• Improvements:

  • Warnings: If verb=1 it prints a warning in case it did not converge (it finished silently until now).
  • Improvements to docs (figures-scaling; intersphinx).
  • Adjust Fields.pha and Fields.amp in accordance with empymod v2: .pha and .amp are now methods; uses directly empymod.utils.EMArray.
  • Adjust tests for empymod v2 (Fields, Fourier).

Data persistence

• New:

  • New functions emg3d.save and emg3d.load to save and load all sort of emg3d instances. The currently implemented backends are h5py for .h5-files (default, but requires h5py to be installed) and numpy for .npz-files.
  • Classes emg3d.utils.Field, emg3d.utils.Model, and emg3d.utils.TensorMesh have new methods .copy(), .to_dict(), and .from_dict().
  • emg3d.utils.Model: Possible to create new models by adding or subtracting existing models, and comparing two models (+, -, == and !=). New attributes shape and size.
  • emg3d.utils.Model does not store the volume any longer (just vnC).

• Deprecations:

  • Deprecated data_write and data_read.

• Internal and bug fixes:

  • All I/O-related stuff moved to its own file io.py.
  • Change from NUMBA_DISABLE_JIT to use py_func for testing and coverage.
  • Bugfix: emg3d.njitted.restrict did not store the {x;y;z}-field if sc_dir was {4;5;6}, respectively.

Sphinx gallery

• Rename solver.solver to solver.solve; load solve also into the main namespace as emg3d.solve.
• Adjustment to termination criterion for STAGNATION: The current error is now compared to the last error of the same cycle type. Together with this the workaround for sslsolver when called with an initial efield introduced in v0.8.0 was removed.
• Adjustment to utils.get_hx_h0 (this might change your boundaries): The calculation domain is now calculated so that the distance for the signal travelling from the source to the boundary and back to the most remote receiver is at least two wavelengths away. If this is within the provided domain, then now extra buffer is added around the domain. Additionally, the function has a new parameter max_domain, which is the maximum distance from the center to the boundary; defaults to 100 km.
• New parameter log for utils.grid2grid; if True, then the interpolation is carried out on a log10-scale.
• Change from the notebook-based emg3d-examples-repo to the sphinx-based emg3d-gallery-repo.

Complex sources

• Strength input for get_source_field can now be complex; it also stores now the source location and its strength and moment.

• get_receiver can now take entire Field instances, and returns in that case (fx, fy, fz) at receiver locations.

• Krylov subspace solvers:

  • Solver now finishes in the middle of preconditioning cycles if tolerance is reached.
  • Solver now aborts if solution diverges or stagnates also for the SSL solvers; it fails and returns a zero field.
  • Removed gmres and lgmres from the supported SSL solvers; they do not work nice for this problem. Supported remain bicgstab (default), cgs, and gcrotmk.

• Various small things:

  • New attribute Field.is_electric, so the field knows if it is electric or magnetic.

  • New verb-possibility: verb=-1 is a continuously updated one-liner, ideal to monitor large sets of calculations or in inversions.

  • The returned info dictionary contains new keys:

    • runtime_at_cycle: accumulated total runtime at each cycle;
    • error_at_cycle: absolute error at each cycle.

  • Simple __repr__ for TensorMesh, Model, Fourier, Time.

• Bugfixes:

  • Related to get_hx_h0, data_write, printing in Fourier.

VolumeModel

• New class VolumeModel; changes in Model:

  • Model now only contains resistivity, magnetic permeability, and electric permittivity.
  • VolumeModel contains the volume-averaged values eta and zeta; called from within emg3d.solver.solver.
  • Full wave equation is enabled again, via epsilon_r; by default it is set to None, hence diffusive approximation.
  • Model parameters are now internally stored as 1D arrays.
  • An {isotropic, VTI, HTI} initiated model can be changed by providing the missing resistivities.

• Bugfix: Up and till version 0.8.1 there was a bug. If resistivity was set with slices, e.g., model.res[:, :, :5]=1e10, it DID NOT update the corresponding eta. This bug was unintentionally fixed in 0.9.0, but only realised now.

• Various:

  • The log now lists the version of emg3d.
  • PEP8: internal imports now use absolute paths instead of relative ones.
  • Move from conda-channel prisae to conda-forge.
  • Automatic deploy for PyPi and conda-forge.