Uncertainty

[katosh]

The core objective of this PR is to introduce uncertainty estimation into Mellon's primary results.

New Features

with_uncertainty Parameter

Integrates a boolean parameter with_uncertainty across all estimators: DensityEstimator, TimeSensitiveDensityEstimator, FunctionEstimator, and DimensionalityEstimator. It modifies the fitted predictor, accessible via the .predict property, to include the following methods:

• .covariance(X): Calculates the (co-)variance of the posterior Gaussian Process (GP).
  • Almost 0 near landmarks; grows for out-of-sample locations.
  • Increases with sparsity.
  • Defaults to diag=True, computing only the covariance matrix diagonal.
• .mean_covariance(X): Computes the (co-)variance through the uncertainty of the mean function's GP posterior.
  • Derived from Bayesian inference for latent density function representation.
  • Increases in low data or low-density areas.
  • Only available with posterior uncertainty quantification, e.g., optimizer='advi' except for the FunctionEstimator where input uncertainty is specified through the sigma parameter.
  • Defaults to diag=True, computing only the covariance matrix diagonal.
• .uncertainty(X): Combines .covariance(X) and .mean_covariance(X).
  • Defaults to diag=True, computing only the covariance matrix diagonal.
  • Square root provides standard deviation.

gp_type Parameter

Introduces the gp_type parameter to all relevant estimators to explicitly specify the Gaussian Process (GP) sparsification strategy, replacing the previously used method argument (with options auto, fixed, and percent) that implicitly controlled sparsification. The available options for gp_type include:

• 'full': Non-sparse GP.
• 'full_nystroem': Sparse GP with Nyström rank reduction, lowering computational complexity.
• 'sparse_cholesky': SParse GP using landmarks/inducing points.
• 'sparse_nystroem': Improved Nyström rank reduction on sparse GP with landmarks, balancing accuracy and efficiency.

This new parameter adds additional validation steps, ensuring that no contradictory parameters are specified. If inconsistencies are detected, a helpful reply guides the user on how to fix the issue. The value can be either a string matching one of the options above or an instance of the mellon.parameters.GaussianProcessType Enum. Partial matches log a warning, using the closest match. Defaults to 'sparse_cholesky'.

Note: Nyström strategies are not applicable to the FunctionEstimator.

y_is_mean Parameter

Adds a boolean parameter y_is_mean to FunctionEstimator, affecting how y values are interpreted:

• Old Behavior: sigma impacted conditional mean functions and predictions.
• Intermediate Behavior: sigma only influenced prediction uncertainty.
• New Parameter: If y_is_mean=True, y values are treated as a fixed mean; sigma reflects only uncertainty. If y_is_mean=False, y is considered a noisy measurement, potentially smoothing values at locations x.

This change benefits DensityEstimator, TimeSensitiveDensityEstimator, and DimensionalityEstimator where function values are predicted for out-of-sample locations after mean GP computation.

check_rank Parameter

Introduces the check_rank parameter to all relevant estimators. This boolean parameter explicitly controls whether the rank check is performed, specifically in the gp_type="sparse_cholesky" case. The rank check assesses the chosen landmarks for adequate complexity by examining the approximate rank of the covariance matrix, issuing a warning if insufficient. Allowed values are:

• True: Always perform the check.
• False: Never perform the check.
• None (Default): Perform the check only if n_landmarks is greater than or equal to n_samples divided by 10.

The default setting aims to bypass unnecessary computation when the number of landmarks is so abundant that insufficient complexity becomes improbable.

normalize Parameter

The normalize parameter is applicable to both the .mean method and .__call__ method within the mellon.Predictor class. When set to True, these methods will subtract log(number of observations) from the value returned. This feature is particularly useful with the DensityEstimator, where normalization adjusts for the number of cells in the training sample, allowing for accurate density comparisons between datasets. This correction takes into account the effect of dataset size, ensuring that differences in total cell numbers are not unduly influential. By default, the parameter is set to False, meaning that density differences due to variations in total cell number will remain uncorrected.

normalize_per_time_point Parameter

This parameter fine-tunes the TimeSensitiveDensityEstimator to handle variations in sampling bias across different time points, ensuring both continuity and differentiability in the resulting density estimation. Notably, it also allows to reflect the growth of a population even if the same number of cells were sampled from each time point.

The normalization is realized by manipulating the nearest neighbor distances
nn_distances to reflect the deviation from an expected cell count.

• Type: Optional, accepts bool, list, array-like, or dict.

Options:

• True: Normalizes to emulate an even distribution of total cell count across all time points.
• False: Retains raw cell counts at each time point for density estimation.
• List/Array-like: Specifies an ordered sequence of total cell count targets for each time point, starting with the earliest.
• Dict: Associates each unique time point with a specific total cell count target.

Notes:

• Relative Metrics: While this parameter adjusts for sample bias, it only requires relative cell counts for comparisons within the dataset; exact counts are not mandatory.
• nn_distance Precedence: If nn_distance is supplied, this parameter will be bypassed, and the provided distances will be used directly.
• The default value is False

Enhancements

• Optimization by saving the intermediate result Lp in the estimators for reuse, enhancing the speed of the predictive function computation in non-Nyström strategies.
• The DimensionalityEstimator.predict now returns a subclass of the mellon.Predictor class instead of a closure. Giving access to serialization and uncertainty computations.
• Expanded testing.
• propagate logging messages and explicit logger name "mellon" everywhere
• extended parameter validation for the estimators now also applies to the compute_L function
• better string representation of estimators and predictors
• bugfix some edge cases
• Revise some documentation (s. b70bb04) and include Predictor page on sphinx doc

Changes

• The mellon.Predictor class now has a method .mean that is an alias to .__call__.
• All mellon.Predictor sub classes ...ConditionalMean... were renamed to ...Conditional... since they now also compute .covariance and .mean_covariance.
• All generating methods for mellon.Predictor were renamed from ...conditional_mean... to conditional.
• A new log message now informs that the normalization is not effective d_method != "fractal". Additionally, using normalize=True in the density predictor triggers a warning that one has to use the non default d_method = "fractal" in the DensityEstimator.

[codecov]

Codecov Report

Attention: 27 lines in your changes are missing coverage. Please review.

Comparison is base (c274b7f) 92.65% compared to head (924aa5a) 97.47%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main       #7      +/-   ##
==========================================
+ Coverage   92.65%   97.47%   +4.81%     
==========================================
  Files          34       35       +1     
  Lines        2547     3518     +971     
==========================================
+ Hits         2360     3429    +1069     
+ Misses        187       89      -98     

Files	Coverage Δ
mellon/__init__.py	100.00% <100.00%> (ø)
mellon/_conditional.py	100.00% <ø> (ø)
mellon/_inference.py	100.00% <ø> (ø)
mellon/_parameters.py	100.00% <ø> (ø)
mellon/_util.py	100.00% <ø> (ø)
mellon/base_cov.py	94.40% <100.00%> (+0.38%)	⬆️
mellon/base_model.py	93.06% <100.00%> (+8.55%)	⬆️
mellon/compute_ls_time.py	100.00% <100.00%> (ø)
mellon/cov.py	100.00% <100.00%> (ø)
mellon/decomposition.py	91.93% <100.00%> (+8.15%)	⬆️
... and 18 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[katosh]

@ManuSetty The refactoring is done. I made the chages I mentioned and I removed the argument method (with options auto, fixed, and percent) in favor of gp_type that lets the user choose the sparsification method explicitly:

    gp_type : str or GaussianProcessType
        The type of sparcification used for the Gaussian Process
         - 'full' None-sparse Gaussian Process
         - 'full_nystroem' Sparse GP with Nyström rank reduction without landmarks,
            which lowers the computational complexity.
         - 'sparse_cholesky' Sparse GP using landmarks/inducing points,
            typically employed to enable scalable GP models.
         - 'sparse_nystroem' Sparse GP using landmarks or inducing points,
            along with an improved Nyström rank reduction method that balances
            accuracy with efficiency.

        The value can be either a string matching one of the above options or an instance of
        the `mellon.parameters.GaussianProcessType` Enum. If a partial match is found with the
        Enum, a warning will be logged, and the closest match will be used.
        Defaults to 'sparse_cholesky'.

This comes with an additional parameter validation making sure no contradictory parameters are specified.

[katosh]

Commit 27b7d63 resolves a major ambiguity harmonizing the new uncertainty computation for the DensityEstimator with the noise input handling of the FunctionEstimator.