Merge remote-tracking branch 'origin/develop' into feature/refactor-v…

…alue

CHANGELOG.md

@@ -30,8 +30,17 @@
 - Extend `NystroemSketchInfluence` with block-diagonal and Gauss-Newton
   approximation
   [PR #596](https://github.com/aai-institute/pyDVL/pull/596)
+- Extend `ArnoldiInfluence` with block-diagonal and Gauss-Newton
+  approximation
+  [PR #598](https://github.com/aai-institute/pyDVL/pull/598)
+- Extend `CgInfluence` with block-diagonal and Gauss-Newton
+  approximation
+  [PR #601](https://github.com/aai-institute/pyDVL/pull/601)
 
-### Fixed
+## Fixed
+- Replace `np.float_` with `np.float64` and `np.alltrue` with `np.all`,
+  as the old aliases are removed in NumPy 2.0
+  [PR #604](https://github.com/aai-institute/pyDVL/pull/604)
 
 - Fix a bug in pydvl.utils.numeric.random_subset where 1 - q was used instead of q
   as the probability of an element being sampled
@@ -61,7 +70,30 @@
     to `regularization` and change the type annotation to allow
     for block-wise regularization parameters
     [PR #596](https://github.com/aai-institute/pyDVL/pull/596)
-
+  - Renaming of parameters of `ArnoldiInfluence`,
+    `hessian_regularization` -> `regularization` (modify type annotation),
+    `rank_estimate` -> `rank`
+    [PR #598](https://github.com/aai-institute/pyDVL/pull/598)
+  - Remove functions remove obsolete functions 
+    `lanczos_low_rank_hessian_approximation`, `model_hessian_low_rank`
+    from `influence.torch.functional`
+    [PR #598](https://github.com/aai-institute/pyDVL/pull/598)
+  - Renaming of parameters of `CgInfluence`,
+    `hessian_regularization` -> `regularization` (modify type annotation),
+    `pre_conditioner` -> `preconditioner`,
+    `use_block_cg` -> `solve_simultaneously`
+    [PR #601](https://github.com/aai-institute/pyDVL/pull/601)
+  - Remove parameter `x0` from `CgInfluence`
+    [PR #601](https://github.com/aai-institute/pyDVL/pull/601)
+  - Rename module 
+    `influence.torch.pre_conditioner` -> `influence.torch.preconditioner`
+    [PR #601](https://github.com/aai-institute/pyDVL/pull/601)
+  - Refactor preconditioner:
+    - renaming `PreConditioner` -> `Preconditioner`
+    - fit to `TensorOperator`
+    [PR #601](https://github.com/aai-institute/pyDVL/pull/601)
+
+
 ## 0.9.2 - 🏗  Bug fixes, logging improvement
 
 ### Added

docs/influence/influence_function_model.md

@@ -23,37 +23,45 @@ gradient method, defined in [@ji_breakdownfree_2017], which solves several
 right hand sides simultaneously.
 
 Optionally, the user can provide a pre-conditioner to improve convergence, such 
-as a [Jacobi pre-conditioner
-][pydvl.influence.torch.pre_conditioner.JacobiPreConditioner], which
+as a [Jacobi preconditioner
+][pydvl.influence.torch.preconditioner.JacobiPreconditioner], which
 is a simple [diagonal pre-conditioner](
 https://en.wikipedia.org/wiki/Preconditioner#Jacobi_(or_diagonal)_preconditioner) 
 based on Hutchinson's diagonal estimator [@bekas_estimator_2007],
-or a [Nyström approximation based pre-conditioner
-][pydvl.influence.torch.pre_conditioner.NystroemPreConditioner], 
-described in [@frangella_randomized_2023]. 
+or a [Nyström approximation based preconditioner
+][pydvl.influence.torch.preconditioner.NystroemPreconditioner], 
+described in [@frangella_randomized_2023].
 
 ```python
-from pydvl.influence.torch import CgInfluence
-from pydvl.influence.torch.pre_conditioner import NystroemPreConditioner
+from pydvl.influence.torch import CgInfluence, BlockMode, SecondOrderMode
+from pydvl.influence.torch.preconditioner import NystroemPreconditioner
 
 if_model = CgInfluence(
     model,
     loss,
-    hessian_regularization=0.0,
+    regularization=0.0,
     rtol=1e-7,
     atol=1e-7,
     maxiter=None,
-    use_block_cg=True,
-    pre_conditioner=NystroemPreConditioner(rank=10)
+    solve_simultaneously=True,
+    preconditioner=NystroemPreconditioner(rank=10),
+    block_structure=BlockMode.FULL,
+    second_order_mode=SecondOrderMode.HESSIAN
 )
 if_model.fit(train_loader)
 ```
 
-The additional optional parameters `rtol`, `atol`, `maxiter`, `use_block_cg` and 
-`pre_conditioner` are respectively, the relative
+The additional optional parameters `rtol`, `atol`, `maxiter`, 
+`solve_simultaneously` and `preconditioner` are respectively, the relative
 tolerance, the absolute tolerance, the maximum number of iterations, 
-a flag indicating whether to use block variant of cg and an optional
-pre-conditioner.
+a flag indicating whether to use a variant of cg to
+simultaneously solving the system for several right hand sides and an optional
+preconditioner.
+
+This implementation is capable of using a block-diagonal
+approximation, see
+[Block-diagonal approximation](#block-diagonal-approximation), and can handle
+[Gauss-Newton approximation](#gauss-newton-approximation).
 
 
 ### Linear time Stochastic Second-Order Approximation (LiSSA)
@@ -78,7 +86,7 @@ from pydvl.influence.torch import LissaInfluence, BlockMode, SecondOrderMode
 if_model = LissaInfluence(
    model,
    loss,
-   regularization=0.0 
+   regularization=0.0, 
    maxiter=1000,
    dampen=0.0,
    scale=10.0,
@@ -114,16 +122,22 @@ the Hessian and \(V\) contains the corresponding eigenvectors. See also
 [@schioppa_scaling_2022].
 
 ```python
-from pydvl.influence.torch import ArnoldiInfluence
+from pydvl.influence.torch import ArnoldiInfluence, BlockMode, SecondOrderMode
 if_model = ArnoldiInfluence(
     model,
     loss,
-    hessian_regularization=0.0,
-    rank_estimate=10,
+    regularization=0.0,
+    rank=10,
     tol=1e-6,
+    block_structure=BlockMode.FULL,
+    second_order_mode=SecondOrderMode.HESSIAN
 )
 if_model.fit(train_loader)
 ```
+This implementation is capable of using a block-matrix
+approximation, see
+[Block-diagonal approximation](#block-diagonal-approximation), and can handle
+[Gauss-Newton approximation](#gauss-newton-approximation).
 
 ### Eigenvalue Corrected K-FAC
 
@@ -201,7 +215,7 @@ see also [@hataya_nystrom_2023] and [@frangella_randomized_2023]. The essential
 parameter is the rank of the approximation.
 
 ```python
-from pydvl.influence.torch import NystroemSketchInfluence
+from pydvl.influence.torch import NystroemSketchInfluence, BlockMode, SecondOrderMode
 if_model = NystroemSketchInfluence(
     model,
     loss,

notebooks/influence_synthetic.ipynb

@@ -848,7 +848,7 @@
     "influence_model = CgInfluence(\n",
     "    model,\n",
     "    F.binary_cross_entropy,\n",
-    "    hessian_regularization=0.0,\n",
+    "    regularization=0.0,\n",
     ")\n",
     "influence_model = influence_model.fit(train_corrupted_data_loader)\n",
     "influence_values = influence_model.influences(\n",

notebooks/support/common.py

@@ -21,16 +21,16 @@
 
 
 def plot_gaussian_blobs(
-    train_ds: Tuple[NDArray[np.float_], NDArray[np.int_]],
-    test_ds: Tuple[NDArray[np.float_], NDArray[np.int_]],
-    x_min: Optional[NDArray[np.float_]] = None,
-    x_max: Optional[NDArray[np.float_]] = None,
+    train_ds: Tuple[NDArray[np.float64], NDArray[np.int_]],
+    test_ds: Tuple[NDArray[np.float64], NDArray[np.int_]],
+    x_min: Optional[NDArray[np.float64]] = None,
+    x_max: Optional[NDArray[np.float64]] = None,
     *,
     xlabel: Optional[str] = None,
     ylabel: Optional[str] = None,
     legend_title: Optional[str] = None,
     vline: Optional[float] = None,
-    line: Optional[NDArray[np.float_]] = None,
+    line: Optional[NDArray[np.float64]] = None,
     suptitle: Optional[str] = None,
     s: Optional[float] = None,
     figsize: Tuple[int, int] = (20, 10),
@@ -104,15 +104,15 @@ def plot_gaussian_blobs(
 
 
 def plot_influences(
-    x: NDArray[np.float_],
-    influences: NDArray[np.float_],
+    x: NDArray[np.float64],
+    influences: NDArray[np.float64],
     corrupted_indices: Optional[List[int]] = None,
     *,
     ax: Optional[plt.Axes] = None,
     xlabel: Optional[str] = None,
     ylabel: Optional[str] = None,
     legend_title: Optional[str] = None,
-    line: Optional[NDArray[np.float_]] = None,
+    line: Optional[NDArray[np.float64]] = None,
     suptitle: Optional[str] = None,
     colorbar_limits: Optional[Tuple] = None,
 ) -> plt.Axes:
@@ -403,7 +403,7 @@ def plot_sample_images(dataset: pd.DataFrame, n_images_per_class: int = 3):
 
 
 def plot_lowest_highest_influence_images(
-    subset_influences: NDArray[np.float_],
+    subset_influences: NDArray[np.float64],
     subset_images: List[JpegImageFile],
     num_to_plot: int,
 ):
@@ -454,7 +454,7 @@ def plot_losses(losses: Losses):
 def corrupt_imagenet(
     dataset: pd.DataFrame,
     fraction_to_corrupt: float,
-    avg_influences: NDArray[np.float_],
+    avg_influences: NDArray[np.float64],
 ) -> Tuple[pd.DataFrame, Dict[Any, List[int]]]:
     """Given the preprocessed tiny imagenet dataset (or a subset of it), 
     it takes a fraction of the images with the highest influence and (randomly)
@@ -494,7 +494,7 @@ def corrupt_imagenet(
 def compute_mean_corrupted_influences(
     corrupted_dataset: pd.DataFrame,
     corrupted_indices: Dict[Any, List[int]],
-    avg_corrupted_influences: NDArray[np.float_],
+    avg_corrupted_influences: NDArray[np.float64],
 ) -> pd.DataFrame:
     """Given a corrupted dataset, it returns a dataframe with average influence for each class,
     separating corrupted and original points.
@@ -534,7 +534,7 @@ def compute_mean_corrupted_influences(
 def plot_corrupted_influences_distribution(
     corrupted_dataset: pd.DataFrame,
     corrupted_indices: Dict[Any, List[int]],
-    avg_corrupted_influences: NDArray[np.float_],
+    avg_corrupted_influences: NDArray[np.float64],
     figsize: Tuple[int, int] = (16, 8),
 ):
     """Given a corrupted dataset, plots the histogram with the distribution of

notebooks/support/types.py

@@ -5,5 +5,5 @@
 
 
 class Losses(NamedTuple):
-    training: NDArray[np.float_]
-    validation: NDArray[np.float_]
+    training: NDArray[np.float64]
+    validation: NDArray[np.float64]

requirements.txt

@@ -1,5 +1,5 @@
 pyDeprecate>=0.3.2
-numpy>=1.20
+numpy>=1.20,<2
 pandas>=1.3
 scikit-learn
 scipy>=1.7.0

src/pydvl/influence/torch/__init__.py

@@ -7,5 +7,5 @@
     LissaInfluence,
     NystroemSketchInfluence,
 )
-from .pre_conditioner import JacobiPreConditioner, NystroemPreConditioner
+from .preconditioner import JacobiPreconditioner, NystroemPreconditioner
 from .util import BlockMode, SecondOrderMode