diff --git a/DifferentiationInterface/README.md b/DifferentiationInterface/README.md index 2bdbc9244..a683cf9f8 100644 --- a/DifferentiationInterface/README.md +++ b/DifferentiationInterface/README.md @@ -17,10 +17,7 @@ An interface to various automatic differentiation (AD) backends in Julia. ## Goal -This package provides a backend-agnostic syntax to differentiate functions of the following types: - -- _one-argument functions_ (allocating): `f(x) = y` -- _two-argument functions_ (mutating): `f!(y, x) = nothing` +This package provides a unified syntax to differentiate functions. ## Features diff --git a/DifferentiationInterface/docs/src/api.md b/DifferentiationInterface/docs/src/api.md index c37435907..24bbc8180 100644 --- a/DifferentiationInterface/docs/src/api.md +++ b/DifferentiationInterface/docs/src/api.md @@ -103,8 +103,7 @@ value_gradient_and_hessian! ```@docs check_available -check_twoarg -check_hessian +check_inplace DifferentiationInterface.outer DifferentiationInterface.inner ``` diff --git a/DifferentiationInterface/docs/src/backends.md b/DifferentiationInterface/docs/src/backends.md index 408b90700..1b08552e6 100644 --- a/DifferentiationInterface/docs/src/backends.md +++ b/DifferentiationInterface/docs/src/backends.md @@ -5,8 +5,9 @@ This page is about the latter, check out [that page](@ref "Operators") to learn ## List of backends -We support all dense backend choices from [ADTypes.jl](https://github.com/SciML/ADTypes.jl): +We support the following dense backend choices from [ADTypes.jl](https://github.com/SciML/ADTypes.jl): +- [`AutoChainRules`](@extref ADTypes.AutoChainRules) - [`AutoDiffractor`](@extref ADTypes.AutoDiffractor) - [`AutoEnzyme`](@extref ADTypes.AutoEnzyme) - [`AutoFastDifferentiation`](@extref ADTypes.AutoFastDifferentiation) @@ -37,11 +38,17 @@ We strongly recommend that users upgrade to Julia 1.10 or above, where all backe ## Features +Given a backend object, you can use: + +- [`check_available`](@ref) to know whether the required AD package is loaded +- [`check_inplace`](@ref) to know whether the backend supports in-place functions (all backends support out-of-place functions) + ```@setup backends using ADTypes using DifferentiationInterface import Markdown +import ChainRulesCore import Diffractor import Enzyme import FastDifferentiation @@ -56,6 +63,7 @@ import Tracker import Zygote backend_examples = [ + AutoChainRules(; ruleconfig=Zygote.ZygoteRuleConfig()), AutoDiffractor(), AutoEnzyme(), AutoFastDifferentiation(), @@ -72,17 +80,16 @@ backend_examples = [ checkmark(x::Bool) = x ? '✅' : '❌' unicode_check_available(backend) = checkmark(check_available(backend)) -unicode_check_hessian(backend) = checkmark(check_hessian(backend; verbose=false)) -unicode_check_twoarg(backend) = checkmark(check_twoarg(backend)) +unicode_check_inplace(backend) = checkmark(check_inplace(backend)) io = IOBuffer() # Table header -println(io, "| Backend | Availability | Two-argument functions | Hessian support |") -println(io, "|:--------|:------------:|:----------------------:|:---------------:|") +println(io, "| Backend | Availability | In-place functions |") +println(io, "|:--------|:------------:|:----------------------:|") for b in backend_examples - join(io, ["`$(nameof(typeof(b)))`", unicode_check_available(b), unicode_check_twoarg(b), unicode_check_hessian(b)], '|') + join(io, ["`$(nameof(typeof(b)))`", unicode_check_available(b), unicode_check_inplace(b)], '|') println(io, '|' ) end backend_table = Markdown.parse(String(take!(io))) @@ -92,21 +99,6 @@ backend_table = Markdown.parse(String(take!(io))) backend_table #hide ``` -### Availability - -You can use [`check_available`](@ref) to verify whether a given backend is loaded. - -### Support for two-argument functions - -All backends are compatible with one-argument functions `f(x) = y`. -Only some are compatible with two-argument functions `f!(y, x) = nothing`. -You can use [`check_twoarg`](@ref) to verify this compatibility. - -### Support for Hessian - -Only some backends are able to compute Hessians. -You can use [`check_hessian`](@ref) to verify this feature (beware that it will try to compute a small Hessian, so it is not instantaneous like the other checks). - ## Backend switch The wrapper [`DifferentiateWith`](@ref) allows you to switch between backends. diff --git a/DifferentiationInterface/docs/src/dev_guide.md b/DifferentiationInterface/docs/src/dev_guide.md index 65afd3f72..b441c31ca 100644 --- a/DifferentiationInterface/docs/src/dev_guide.md +++ b/DifferentiationInterface/docs/src/dev_guide.md @@ -24,8 +24,7 @@ Most operators have 4 variants, which look like this in the first order: `operat ## New operator To implement a new operator for an existing backend, you need to write 5 methods: 1 for [preparation](@ref Preparation) and 4 corresponding to the variants of the operator (see above). -In some cases, a subset of those methods will be enough, but most of the time, forgetting one will trigger errors. -For first-order operators, you may also want to support [two-argument functions](@ref "Mutation and signatures"), which requires another 5 methods (defined on `f!` instead of `f`). +For first-order operators, you may also want to support [in-place functions](@ref "Mutation and signatures"), which requires another 5 methods (defined on `f!` instead of `f`). The method `prepare_operator` must output an `extras` object of the correct type. For instance, `prepare_gradient(f, backend, x)` must return a [`DifferentiationInterface.GradientExtras`](@ref). @@ -40,7 +39,7 @@ Your AD package needs to be registered first. ### Core code In the main package, you should define a new struct `SuperDiffBackend` which subtypes [`ADTypes.AbstractADType`](@extref ADTypes), and endow it with the fields you need to parametrize your differentiation routines. -You also have to define [`ADTypes.mode`](@extref) and [`DifferentiationInterface.twoarg_support`](@ref) on `SuperDiffBackend`. +You also have to define [`ADTypes.mode`](@extref) and [`DifferentiationInterface.inplace_support`](@ref) on `SuperDiffBackend`. !!! info In the end, this backend struct will need to be contributed to [ADTypes.jl](https://github.com/SciML/ADTypes.jl). @@ -79,5 +78,4 @@ GROUP = get(ENV, "JULIA_DI_TEST_GROUP", "Back/SuperDiff") but don't forget to switch it back before pushing. -Finally, you need to add your backend to the documentation, modifying every page that involves a list of backends. -That includes the README. +Finally, you need to add your backend to the documentation, modifying every page that involves a list of backends (including the `README.md`). \ No newline at end of file diff --git a/DifferentiationInterface/docs/src/implementations.md b/DifferentiationInterface/docs/src/implementations.md index 64794d022..5ecd7e9ea 100644 --- a/DifferentiationInterface/docs/src/implementations.md +++ b/DifferentiationInterface/docs/src/implementations.md @@ -1,10 +1,6 @@ # Implementations -DifferentiationInterface.jl provides a handful of [operators](@ref "Operators") like [`gradient`](@ref) or [`jacobian`](@ref), each with several variants: - -- **out-of-place** or **in-place** behavior -- **with** or **without primal** output value -- support for **one-argument functions** `y = f(x)` or **two-argument functions** `f!(y, x)` +DifferentiationInterface.jl provides a handful of [operators](@ref "Operators") like [`gradient`](@ref) or [`jacobian`](@ref), each with several variants: out-of-place or in-place behavior, with or without primal output value. While it is possible to define every operator using just [`pushforward`](@ref) and [`pullback`](@ref), some backends have more efficient implementations of high-level operators. When they are available, we nearly always call these backend-specific overloads. @@ -24,7 +20,7 @@ The cells can have three values: ```@setup overloads using ADTypes: AbstractADType using DifferentiationInterface -using DifferentiationInterface: twoarg_support, TwoArgSupported +using DifferentiationInterface: inplace_support, InPlaceSupported using Markdown: Markdown using Diffractor: Diffractor @@ -152,16 +148,16 @@ function print_overloads(backend, ext::Symbol) io = IOBuffer() ext = Base.get_extension(DifferentiationInterface, ext) - println(io, "#### One-argument functions `y = f(x)`") + println(io, "#### Out-of-place functions `f(x) = y`") println(io) print_overload_table(io, operators_and_types_f(backend), ext) - println(io, "#### Two-argument functions `f!(y, x)`") + println(io, "#### In-place functions `f!(y, x) = nothing`") println(io) - if twoarg_support(backend) == TwoArgSupported() + if inplace_support(backend) == InPlaceSupported() print_overload_table(io, operators_and_types_f!(backend), ext) else - println(io, "Backend doesn't support mutating functions.") + println(io, "Backend doesn't support in-place functions.") end return Markdown.parse(String(take!(io))) diff --git a/DifferentiationInterface/docs/src/operators.md b/DifferentiationInterface/docs/src/operators.md index 080ae0cc9..96cabbec8 100644 --- a/DifferentiationInterface/docs/src/operators.md +++ b/DifferentiationInterface/docs/src/operators.md @@ -14,7 +14,7 @@ Given a function `f(x) = y`, there are several differentiation operators availab Below we list and describe all the operators we support. !!! tip - See the book [The Elements of Differentiable Programming](https://arxiv.org/abs/2403.14606) for details on these concepts. + Read the book [The Elements of Differentiable Programming](https://arxiv.org/abs/2403.14606) for details on these concepts. ### High-level operators @@ -35,15 +35,18 @@ These operators are computed using the input `x` and a "seed" `v`, which lives e - in the same space as `x` (we call it `dx`) - or in the same space as `y` (we call it `dy`) -| operator | order | input `x` | output `y` | seed `v` | operator result type | operator result shape | -| :----------------------------- | :---- | :-------------- | :----------- | :------- | :------------------- | :-------------------- | -| [`pushforward`](@ref) (or JVP) | 1 | `Any` | `Any` | `dx` | same as `y` | `size(y)` | -| [`pullback`](@ref) (or VJP) | 1 | `Any` | `Any` | `dy` | same as `x` | `size(x)` | -| [`hvp`](@ref) | 2 | `AbstractArray` | `Number` | `dx` | same as `x` | `size(x)` | +| operator | order | input `x` | output `y` | seed `v` | operator result type | operator result shape | +| :-------------------------- | :---- | :-------------- | :----------- | :------- | :------------------- | :-------------------- | +| [`pushforward`](@ref) (JVP) | 1 | `Any` | `Any` | `dx` | same as `y` | `size(y)` | +| [`pullback`](@ref) (VJP) | 1 | `Any` | `Any` | `dy` | same as `x` | `size(x)` | +| [`hvp`](@ref) | 2 | `AbstractArray` | `Number` | `dx` | same as `x` | `size(x)` | ## Variants -Several variants of each operator are defined. +Several variants of each operator are defined: + +- out-of-place operators return a new derivative object +- in-place operators mutate the provided derivative object | out-of-place | in-place | out-of-place + primal | in-place + primal | | :-------------------------- | :--------------------------- | :----------------------------------------------- | :------------------------------------------------ | @@ -58,25 +61,25 @@ Several variants of each operator are defined. ## Mutation and signatures -Some operators support two types of functions: +Two kinds of functions are supported: -- one-argument functions `f(x) = y` -- two-argument functions `f!(y, x) = nothing` +- out-of-place functions `f(x) = y` +- in-place functions `f!(y, x) = nothing` !!! warning - Only [`pushforward`](@ref), [`pullback`](@ref), [`derivative`](@ref) and [`jacobian`](@ref) support two-argument functions at the moment. + In-place functions only work with [`pushforward`](@ref), [`pullback`](@ref), [`derivative`](@ref) and [`jacobian`](@ref). -The same operators are defined for both cases, but they have different signatures (they take different arguments): +This results in various operator signatures (the necessary arguments and their order): -| signature | out-of-place | in-place | -| :--------- | :------------------------------------------- | :---------------------------------------------------- | -| `f(x)` | `operator(f, [extras,] backend, x, [v])` | `operator!(f, result, [extras,] backend, x, [v])` | -| `f!(y, x)` | `operator(f!, y, [extras,] backend, x, [v])` | `operator!(f!, y, result, [extras,] backend, x, [v])` | +| function signature | out-of-place operator | in-place operator | +| :-------------------- | :--------------------------- | :------------------------------------ | +| out-of-place function | `op(f, backend, x, [v])` | `op!(f, result, backend, x, [v])` | +| in-place function | `op(f!, y, backend, x, [v])` | `op!(f!, y, result, backend, x, [v])` | !!! warning The positional arguments between `f`/`f!` and `backend` are always mutated. This convention holds regardless of the bang `!` in the operator name. - In particular, for two-argument functions `f!(y, x)`, every variant of every operator will mutate `y`. + In particular, for in-place functions `f!(y, x)`, every variant of every operator will mutate `y`. ## Preparation @@ -85,30 +88,30 @@ The same operators are defined for both cases, but they have different signature In many cases, AD can be accelerated if the function has been called at least once (e.g. to record a tape) or if some cache objects are provided. This preparation procedure is backend-specific, but we expose a common syntax to achieve it. -| operator | preparation function (different point) | preparation function (same point) | -| :------------------ | :------------------------------------- | :--------------------------------------- | -| `derivative` | [`prepare_derivative`](@ref) | - | -| `gradient` | [`prepare_gradient`](@ref) | - | -| `jacobian` | [`prepare_jacobian`](@ref) | - | -| `second_derivative` | [`prepare_second_derivative`](@ref) | - | -| `hessian` | [`prepare_hessian`](@ref) | - | -| `pushforward` | [`prepare_pushforward`](@ref) | [`prepare_pushforward_same_point`](@ref) | -| `pullback` | [`prepare_pullback`](@ref) | [`prepare_pullback_same_point`](@ref) | -| `hvp` | [`prepare_hvp`](@ref) | [`prepare_hvp_same_point`](@ref) | +| operator | preparation (different point) | preparation (same point) | +| :------------------ | :---------------------------------- | :--------------------------------------- | +| `derivative` | [`prepare_derivative`](@ref) | - | +| `gradient` | [`prepare_gradient`](@ref) | - | +| `jacobian` | [`prepare_jacobian`](@ref) | - | +| `second_derivative` | [`prepare_second_derivative`](@ref) | - | +| `hessian` | [`prepare_hessian`](@ref) | - | +| `pushforward` | [`prepare_pushforward`](@ref) | [`prepare_pushforward_same_point`](@ref) | +| `pullback` | [`prepare_pullback`](@ref) | [`prepare_pullback_same_point`](@ref) | +| `hvp` | [`prepare_hvp`](@ref) | [`prepare_hvp_same_point`](@ref) | In addition, the preparation syntax depends on the number of arguments accepted by the function. -| signature | preparation signature | -| :--------- | :----------------------------------------- | -| `f(x)` | `prepare_operator(f, backend, x, [v])` | -| `f!(y, x)` | `prepare_operator(f!, y, backend, x, [v])` | +| function signature | preparation signature | +| :-------------------- | :----------------------------------- | +| out-of-place function | `prepare_op(f, backend, x, [v])` | +| in-place function | `prepare_op(f!, y, backend, x, [v])` | Preparation creates an object called `extras` which contains the the necessary information to speed up an operator and its variants. The idea is that you prepare only once, which can be costly, but then call the operator several times while reusing the same `extras`. ```julia -operator(f, backend, x, [v]) # slow because it includes preparation -operator(f, extras, backend, x, [v]) # fast because it skips preparation +op(f, backend, x, [v]) # slow because it includes preparation +op(f, extras, backend, x, [v]) # fast because it skips preparation ``` !!! warning @@ -119,11 +122,11 @@ operator(f, extras, backend, x, [v]) # fast because it skips preparation Deciding whether it is safe to reuse the results of preparation is not easy. Here are the general rules that we strive to implement: -| | different point | same point | -| :------------------------ | :------------------------------------------- | :------------------------------------------- | -| the output `extras` of... | `prepare_operator(f, b, x)` | `prepare_operator_same_point(f, b, x, v)` | -| can be used in... | `operator(f, extras, b, other_x)` | `operator(f, extras, b, x, other_v)` | -| provided that... | `other_x` has the same type and shape as `x` | `other_v` has the same type and shape as `v` | +| | different point | same point | +| :------------------------ | :--------------------------------------- | :--------------------------------------- | +| the output `extras` of... | `prepare_op(f, b, x)` | `prepare_op_same_point(f, b, x, v)` | +| can be used in... | `op(f, extras, b, other_x)` | `op(f, extras, b, x, other_v)` | +| provided that... | `other_x` has same type and shape as `x` | `other_v` has same type and shape as `v` | These rules hold for the majority of backends, but there are some exceptions: see [this page](@ref "Preparation") to know more. @@ -153,12 +156,9 @@ Usually, the most efficient approach for Hessians is forward-over-reverse, i.e. !!! danger `SecondOrder` backends do not support first-order operators. -!!! warning - Preparation does not yet work for the inner differentiation step of a `SecondOrder`, only the outer differentiation is prepared. - ## Sparsity -When computing sparse Jacobians or Hessians, it is possible to take advantage of their sparsity pattern to speed things up. +When computing sparse Jacobians or Hessians (with a lot of zeros in the matrix), it is possible to take advantage of their sparsity pattern to speed things up. For this to work, three ingredients are needed (read [this survey](https://epubs.siam.org/doi/10.1137/S0036144504444711) to understand why): 1. An underlying (dense) backend @@ -172,8 +172,8 @@ For this to work, three ingredients are needed (read [this survey](https://epubs Generic sparse AD is now located in a package extension which depends on SparseMatrixColorings.jl. These ingredients can be combined within the [`AutoSparse`](@extref ADTypes.AutoSparse) wrapper, which DifferentiationInterface.jl re-exports. -Note that for sparse Hessians, you need to put the `SecondOrder` backend inside `AutoSparse`, and not the other way around. `AutoSparse` backends only support operators [`jacobian`](@ref) and [`hessian`](@ref) (as well as their variants). +Note that for sparse Hessians, you need to put the `SecondOrder` backend inside `AutoSparse`, and not the other way around. The preparation step of `jacobian` or `hessian` with an `AutoSparse` backend can be long, because it needs to detect the sparsity pattern and color the resulting sparse matrix. But after preparation, the more zeros are present in the matrix, the greater the speedup will be compared to dense differentiation. @@ -197,8 +197,4 @@ We voluntarily keep the type annotations minimal, so that passing more complex o ### Multiple inputs/outputs Restricting the API to one input and one output has many coding advantages, but it is not very flexible. -If you need more than that, use [ComponentArrays.jl](https://github.com/jonniedie/ComponentArrays.jl) to wrap several objects inside a single `ComponentVector`. - -### Batched evaluation - -This is not supported at the moment, but we plan to allow several seeds at once in low-level operators (similar to the chunking in ForwardDiff.jl or the batches in Enzyme.jl). +If you need more than that, try using [ComponentArrays.jl](https://github.com/jonniedie/ComponentArrays.jl) to wrap several objects inside a single `ComponentVector`. diff --git a/DifferentiationInterface/docs/src/tutorial1.md b/DifferentiationInterface/docs/src/tutorial1.md index afd386673..84df6a7b2 100644 --- a/DifferentiationInterface/docs/src/tutorial1.md +++ b/DifferentiationInterface/docs/src/tutorial1.md @@ -12,14 +12,10 @@ A common use case of automatic differentiation (AD) is optimizing real-valued fu Let's define a simple objective (the squared norm) and a random input vector ```@example tuto1 -function f(x::AbstractVector{T}) where {T} - y = zero(T) - for i in eachindex(x) - y += abs2(x[i]) - end - return y -end +f(x) = sum(abs2, x) +``` +```@example tuto1 x = collect(1.0:5.0) ``` @@ -67,10 +63,10 @@ grad # has been mutated ``` The bang indicates that one of the arguments of `gradient!` might be mutated. -More precisely, our convention is that _every positional argument between the function and the backend is mutated (and the `extras` too, see below)_. +More precisely, our convention is that _every positional argument between the function and the backend is mutated_. ```@example tuto1 -@benchmark gradient!($f, _grad, $backend, $x) evals=1 setup=(_grad=similar($x)) +@benchmark gradient!($f, $grad, $backend, $x) ``` For some reason the in-place version is not much better than your first attempt. @@ -84,7 +80,7 @@ These objects can be reused between gradient computations, even on different inp We abstract away the preparation step behind a backend-agnostic syntax: ```@example tuto1 -extras = prepare_gradient(f, backend, randn(eltype(x), size(x))) +extras = prepare_gradient(f, backend, zero(x)) ``` You don't need to know what this object is, you just need to pass it to the gradient operator. @@ -100,10 +96,7 @@ grad # has been mutated Preparation makes the gradient computation much faster, and (in this case) allocation-free. ```@example tuto1 -@benchmark gradient!($f, _grad, _extras, $backend, $x) evals=1 setup=( - _grad=similar($x); - _extras=prepare_gradient($f, $backend, $x) -) +@benchmark gradient!($f, $grad, $extras, $backend, $x) ``` Beware that the `extras` object is nearly always mutated by differentiation operators, even though it is given as the last positional argument. @@ -128,10 +121,9 @@ gradient(f, backend2, x) And you can run the same benchmarks to see what you gained (although such a small input may not be realistic): ```@example tuto1 -@benchmark gradient!($f, _grad, _extras, $backend2, $x) evals=1 setup=( - _grad=similar($x); - _extras=prepare_gradient($f, $backend2, $x) -) +extras2 = prepare_gradient(f, backend2, zero(x)) + +@benchmark gradient!($f, $grad, $extras2, $backend2, $x) ``` In short, DifferentiationInterface.jl allows for easy testing and comparison of AD backends. diff --git a/DifferentiationInterface/docs/src/tutorial2.md b/DifferentiationInterface/docs/src/tutorial2.md index 5543cd662..4c9cad723 100644 --- a/DifferentiationInterface/docs/src/tutorial2.md +++ b/DifferentiationInterface/docs/src/tutorial2.md @@ -17,18 +17,24 @@ f_sparse_scalar(x::AbstractVector) = sum(f_sparse_vector(x) .^ 2) nothing # hide ``` +Let's also pick a random test vector. + +```@example tuto2 +x = float.(1:8); +``` + ## Dense backends When we use the [`jacobian`](@ref) or [`hessian`](@ref) operator with a dense backend, we get a dense matrix with plenty of zeros. ```@example tuto2 dense_first_order_backend = AutoForwardDiff() -J_dense = jacobian(f_sparse_vector, dense_first_order_backend, float.(1:10)) +J_dense = jacobian(f_sparse_vector, dense_first_order_backend, x) ``` ```@example tuto2 dense_second_order_backend = SecondOrder(AutoForwardDiff(), AutoZygote()) -H_dense = hessian(f_sparse_scalar, dense_second_order_backend, float.(1:10)) +H_dense = hessian(f_sparse_scalar, dense_second_order_backend, x) ``` The results are correct but the procedure is very slow. @@ -36,7 +42,7 @@ By using a sparse backend, we can get the runtime to increase with the number of ## Sparse backends -Recipe to create a sparse backend: combine a dense backend, a sparsity detector and a coloring algorithm inside [`AutoSparse`](@extref ADTypes.AutoSparse). +Recipe to create a sparse backend: combine a dense backend, a sparsity detector and a compatible coloring algorithm inside [`AutoSparse`](@extref ADTypes.AutoSparse). The following are reasonable defaults: ```@example tuto2 @@ -60,11 +66,11 @@ nothing # hide Now the resulting matrices are sparse: ```@example tuto2 -jacobian(f_sparse_vector, sparse_first_order_backend, float.(1:10)) +jacobian(f_sparse_vector, sparse_first_order_backend, x) ``` ```@example tuto2 -hessian(f_sparse_scalar, sparse_second_order_backend, float.(1:10)) +hessian(f_sparse_scalar, sparse_second_order_backend, x) ``` ## Sparse preparation @@ -76,15 +82,15 @@ The speedup becomes very visible in large dimensions. ```@example tuto2 n = 1000 -jac_extras_dense = prepare_jacobian(f_sparse_vector, dense_first_order_backend, randn(n)); -jac_extras_sparse = prepare_jacobian(f_sparse_vector, sparse_first_order_backend, randn(n)); +jac_extras_dense = prepare_jacobian(f_sparse_vector, dense_first_order_backend, zeros(n)) +jac_extras_sparse = prepare_jacobian(f_sparse_vector, sparse_first_order_backend, zeros(n)) nothing # hide ``` ```@example tuto2 -@benchmark jacobian($f_sparse_vector, $jac_extras_dense, $dense_first_order_backend, $(randn(n))) evals=1 +@benchmark jacobian($f_sparse_vector, $jac_extras_dense, $dense_first_order_backend, $(randn(n))) ``` ```@example tuto2 -@benchmark jacobian($f_sparse_vector, $jac_extras_sparse, $sparse_first_order_backend, $(randn(n))) evals=1 +@benchmark jacobian($f_sparse_vector, $jac_extras_sparse, $sparse_first_order_backend, $(randn(n))) ``` diff --git a/DifferentiationInterface/ext/DifferentiationInterfaceChainRulesCoreExt/DifferentiationInterfaceChainRulesCoreExt.jl b/DifferentiationInterface/ext/DifferentiationInterfaceChainRulesCoreExt/DifferentiationInterfaceChainRulesCoreExt.jl index 3aa6c2cc3..0f16bcaca 100644 --- a/DifferentiationInterface/ext/DifferentiationInterfaceChainRulesCoreExt/DifferentiationInterfaceChainRulesCoreExt.jl +++ b/DifferentiationInterface/ext/DifferentiationInterfaceChainRulesCoreExt/DifferentiationInterfaceChainRulesCoreExt.jl @@ -20,7 +20,7 @@ const AutoForwardChainRules = AutoChainRules{<:RuleConfig{>:HasForwardsMode}} const AutoReverseChainRules = AutoChainRules{<:RuleConfig{>:HasReverseMode}} DI.check_available(::AutoChainRules) = true -DI.twoarg_support(::AutoChainRules) = DI.TwoArgNotSupported() +DI.inplace_support(::AutoChainRules) = DI.InPlaceNotSupported() include("reverse_onearg.jl") include("differentiate_with.jl") diff --git a/DifferentiationInterface/ext/DifferentiationInterfaceDiffractorExt/DifferentiationInterfaceDiffractorExt.jl b/DifferentiationInterface/ext/DifferentiationInterfaceDiffractorExt/DifferentiationInterfaceDiffractorExt.jl index 6052b65d2..adcc3cfb4 100644 --- a/DifferentiationInterface/ext/DifferentiationInterfaceDiffractorExt/DifferentiationInterfaceDiffractorExt.jl +++ b/DifferentiationInterface/ext/DifferentiationInterfaceDiffractorExt/DifferentiationInterfaceDiffractorExt.jl @@ -6,7 +6,7 @@ using DifferentiationInterface: NoPushforwardExtras, Tangents using Diffractor: DiffractorRuleConfig, TaylorTangentIndex, ZeroBundle, bundle, ∂☆ DI.check_available(::AutoDiffractor) = true -DI.twoarg_support(::AutoDiffractor) = DI.TwoArgNotSupported() +DI.inplace_support(::AutoDiffractor) = DI.InPlaceNotSupported() DI.pullback_performance(::AutoDiffractor) = DI.PullbackSlow() ## Pushforward diff --git a/DifferentiationInterface/ext/DifferentiationInterfaceFiniteDifferencesExt/DifferentiationInterfaceFiniteDifferencesExt.jl b/DifferentiationInterface/ext/DifferentiationInterfaceFiniteDifferencesExt/DifferentiationInterfaceFiniteDifferencesExt.jl index 97d7f7546..984d49a52 100644 --- a/DifferentiationInterface/ext/DifferentiationInterfaceFiniteDifferencesExt/DifferentiationInterfaceFiniteDifferencesExt.jl +++ b/DifferentiationInterface/ext/DifferentiationInterfaceFiniteDifferencesExt/DifferentiationInterfaceFiniteDifferencesExt.jl @@ -9,7 +9,7 @@ using FiniteDifferences: FiniteDifferences, grad, jacobian, jvp, j′vp using LinearAlgebra: dot DI.check_available(::AutoFiniteDifferences) = true -DI.twoarg_support(::AutoFiniteDifferences) = DI.TwoArgNotSupported() +DI.inplace_support(::AutoFiniteDifferences) = DI.InPlaceNotSupported() ## Pushforward diff --git a/DifferentiationInterface/ext/DifferentiationInterfaceTrackerExt/DifferentiationInterfaceTrackerExt.jl b/DifferentiationInterface/ext/DifferentiationInterfaceTrackerExt/DifferentiationInterfaceTrackerExt.jl index 1efd00c58..026ada6bb 100644 --- a/DifferentiationInterface/ext/DifferentiationInterfaceTrackerExt/DifferentiationInterfaceTrackerExt.jl +++ b/DifferentiationInterface/ext/DifferentiationInterfaceTrackerExt/DifferentiationInterfaceTrackerExt.jl @@ -7,7 +7,7 @@ using Tracker: Tracker, back, data, forward, gradient, jacobian, param, withgrad using Compat DI.check_available(::AutoTracker) = true -DI.twoarg_support(::AutoTracker) = DI.TwoArgNotSupported() +DI.inplace_support(::AutoTracker) = DI.InPlaceNotSupported() ## Pullback diff --git a/DifferentiationInterface/ext/DifferentiationInterfaceZygoteExt/DifferentiationInterfaceZygoteExt.jl b/DifferentiationInterface/ext/DifferentiationInterfaceZygoteExt/DifferentiationInterfaceZygoteExt.jl index d0013e1ff..c9a3e6718 100644 --- a/DifferentiationInterface/ext/DifferentiationInterfaceZygoteExt/DifferentiationInterfaceZygoteExt.jl +++ b/DifferentiationInterface/ext/DifferentiationInterfaceZygoteExt/DifferentiationInterfaceZygoteExt.jl @@ -17,7 +17,7 @@ using Zygote: using Compat DI.check_available(::AutoZygote) = true -DI.twoarg_support(::AutoZygote) = DI.TwoArgNotSupported() +DI.inplace_support(::AutoZygote) = DI.InPlaceNotSupported() ## Pullback diff --git a/DifferentiationInterface/src/DifferentiationInterface.jl b/DifferentiationInterface/src/DifferentiationInterface.jl index 588000c83..70659cec5 100644 --- a/DifferentiationInterface/src/DifferentiationInterface.jl +++ b/DifferentiationInterface/src/DifferentiationInterface.jl @@ -102,11 +102,11 @@ export prepare_hvp, prepare_hvp_same_point export prepare_derivative, prepare_gradient, prepare_jacobian export prepare_second_derivative, prepare_hessian -export check_available, check_twoarg, check_hessian - export DifferentiateWith export DenseSparsityDetector +export check_available, check_inplace + ## Re-exported from ADTypes export AutoChainRules @@ -127,7 +127,6 @@ export AutoSparse ## Public but not exported -@compat public inner -@compat public outer +@compat public inner, outer end # module diff --git a/DifferentiationInterface/src/first_order/derivative.jl b/DifferentiationInterface/src/first_order/derivative.jl index 1f0d505bf..0747ab2fd 100644 --- a/DifferentiationInterface/src/first_order/derivative.jl +++ b/DifferentiationInterface/src/first_order/derivative.jl @@ -8,7 +8,7 @@ Create an `extras` object that can be given to [`derivative`](@ref) and its vari !!! warning If the function changes in any way, the result of preparation will be invalidated, and you will need to run it again. - In the two-argument case, `y` is mutated by `f!` during preparation. + For in-place functions, `y` is mutated by `f!` during preparation. """ function prepare_derivative end diff --git a/DifferentiationInterface/src/first_order/jacobian.jl b/DifferentiationInterface/src/first_order/jacobian.jl index bf4011cea..6c55b1aa4 100644 --- a/DifferentiationInterface/src/first_order/jacobian.jl +++ b/DifferentiationInterface/src/first_order/jacobian.jl @@ -8,7 +8,7 @@ Create an `extras` object that can be given to [`jacobian`](@ref) and its varian !!! warning If the function changes in any way, the result of preparation will be invalidated, and you will need to run it again. - In the two-argument case, `y` is mutated by `f!` during preparation. + For in-place functions, `y` is mutated by `f!` during preparation. """ function prepare_jacobian end diff --git a/DifferentiationInterface/src/first_order/pullback.jl b/DifferentiationInterface/src/first_order/pullback.jl index 9613b6ca8..7491bb126 100644 --- a/DifferentiationInterface/src/first_order/pullback.jl +++ b/DifferentiationInterface/src/first_order/pullback.jl @@ -8,7 +8,7 @@ Create an `extras` object that can be given to [`pullback`](@ref) and its varian !!! warning If the function changes in any way, the result of preparation will be invalidated, and you will need to run it again. - In the two-argument case, `y` is mutated by `f!` during preparation. + For in-place functions, `y` is mutated by `f!` during preparation. """ function prepare_pullback end @@ -20,7 +20,7 @@ Create an `extras_same` object that can be given to [`pullback`](@ref) and its v !!! warning If the function or the point changes in any way, the result of preparation will be invalidated, and you will need to run it again. - In the two-argument case, `y` is mutated by `f!` during preparation. + For in-place functions, `y` is mutated by `f!` during preparation. """ function prepare_pullback_same_point end diff --git a/DifferentiationInterface/src/first_order/pushforward.jl b/DifferentiationInterface/src/first_order/pushforward.jl index 9892ded64..957791ff3 100644 --- a/DifferentiationInterface/src/first_order/pushforward.jl +++ b/DifferentiationInterface/src/first_order/pushforward.jl @@ -8,7 +8,7 @@ Create an `extras` object that can be given to [`pushforward`](@ref) and its var !!! warning If the function changes in any way, the result of preparation will be invalidated, and you will need to run it again. - In the two-argument case, `y` is mutated by `f!` during preparation. + For in-place functions, `y` is mutated by `f!` during preparation. """ function prepare_pushforward end @@ -20,7 +20,7 @@ Create an `extras_same` object that can be given to [`pushforward`](@ref) and it !!! warning If the function or the point changes in any way, the result of preparation will be invalidated, and you will need to run it again. - In the two-argument case, `y` is mutated by `f!` during preparation. + For in-place functions, `y` is mutated by `f!` during preparation. """ function prepare_pushforward_same_point end diff --git a/DifferentiationInterface/src/misc/differentiate_with.jl b/DifferentiationInterface/src/misc/differentiate_with.jl index 4a2d5ad79..36badfc8c 100644 --- a/DifferentiationInterface/src/misc/differentiate_with.jl +++ b/DifferentiationInterface/src/misc/differentiate_with.jl @@ -7,7 +7,7 @@ This works by defining new rules overriding the behavior of the outer backend th !!! warning This is an experimental functionality, whose API cannot yet be considered stable. - At the moment, it only supports one-argument functions, and rules are only defined for [ChainRules.jl](https://github.com/JuliaDiff/ChainRules.jl)-compatible outer backends. + It only supports out-of-place functions, and rules are only defined for [ChainRules.jl](https://github.com/JuliaDiff/ChainRules.jl)-compatible outer backends. # Fields diff --git a/DifferentiationInterface/src/misc/from_primitive.jl b/DifferentiationInterface/src/misc/from_primitive.jl index c51ed0c61..e5ff36866 100644 --- a/DifferentiationInterface/src/misc/from_primitive.jl +++ b/DifferentiationInterface/src/misc/from_primitive.jl @@ -9,7 +9,7 @@ function multibasis(fromprim::FromPrimitive, x::AbstractArray, inds) end check_available(fromprim::FromPrimitive) = check_available(fromprim.backend) -twoarg_support(fromprim::FromPrimitive) = twoarg_support(fromprim.backend) +inplace_support(fromprim::FromPrimitive) = inplace_support(fromprim.backend) function pick_batchsize(fromprim::FromPrimitive, dimension::Integer) return pick_batchsize(fromprim.backend, dimension) diff --git a/DifferentiationInterface/src/misc/zero_backends.jl b/DifferentiationInterface/src/misc/zero_backends.jl index 71a2083a9..ff1c7af4d 100644 --- a/DifferentiationInterface/src/misc/zero_backends.jl +++ b/DifferentiationInterface/src/misc/zero_backends.jl @@ -18,7 +18,7 @@ struct AutoZeroForward <: AbstractADType end ADTypes.mode(::AutoZeroForward) = ForwardMode() check_available(::AutoZeroForward) = true -twoarg_support(::AutoZeroForward) = TwoArgSupported() +inplace_support(::AutoZeroForward) = InPlaceSupported() prepare_pushforward(f, ::AutoZeroForward, x, tx::Tangents) = NoPushforwardExtras() prepare_pushforward(f!, y, ::AutoZeroForward, x, tx::Tangents) = NoPushforwardExtras() @@ -71,7 +71,7 @@ struct AutoZeroReverse <: AbstractADType end ADTypes.mode(::AutoZeroReverse) = ReverseMode() check_available(::AutoZeroReverse) = true -twoarg_support(::AutoZeroReverse) = TwoArgSupported() +inplace_support(::AutoZeroReverse) = InPlaceSupported() prepare_pullback(f, ::AutoZeroReverse, x, ty::Tangents) = NoPullbackExtras() prepare_pullback(f!, y, ::AutoZeroReverse, x, ty::Tangents) = NoPullbackExtras() diff --git a/DifferentiationInterface/src/utils/check.jl b/DifferentiationInterface/src/utils/check.jl index 00efa28ac..54bac8b1e 100644 --- a/DifferentiationInterface/src/utils/check.jl +++ b/DifferentiationInterface/src/utils/check.jl @@ -12,33 +12,8 @@ end check_available(backend::AutoSparse) = check_available(dense_ad(backend)) """ - check_twoarg(backend) + check_inplace(backend) -Check whether `backend` supports differentiation of two-argument functions. +Check whether `backend` supports differentiation of in-place functions. """ -check_twoarg(backend::AbstractADType) = Bool(twoarg_support(backend)) - -hess_checker(x::AbstractArray) = x[1] * x[1] * x[2] * x[2] - -""" - check_hessian(backend) - -Check whether `backend` supports second order differentiation by trying to compute a hessian. - -!!! warning - Might take a while due to compilation time. -""" -function check_hessian(backend::AbstractADType; verbose=true) - try - x = [2.0, 3.0] - hess = hessian(hess_checker, backend, x) - hess_th = [ - 2*abs2(x[2]) 4*x[1]*x[2] - 4*x[1]*x[2] 2*abs2(x[1]) - ] - return isapprox(hess, hess_th; rtol=1e-3) - catch exception - verbose && @warn "Backend $backend does not support hessian" exception - return false - end -end +check_inplace(backend::AbstractADType) = Bool(inplace_support(backend)) diff --git a/DifferentiationInterface/src/utils/traits.jl b/DifferentiationInterface/src/utils/traits.jl index 1e3c3b626..b1f65c117 100644 --- a/DifferentiationInterface/src/utils/traits.jl +++ b/DifferentiationInterface/src/utils/traits.jl @@ -1,37 +1,37 @@ ## Mutation -abstract type MutationBehavior end +abstract type InPlaceBehavior end """ - TwoArgSupported + InPlaceSupported -Trait identifying backends that support two-argument functions `f!(y, x)`. +Trait identifying backends that support in-place functions `f!(y, x)`. """ -struct TwoArgSupported <: MutationBehavior end +struct InPlaceSupported <: InPlaceBehavior end """ - TwoArgNotSupported + InPlaceNotSupported -Trait identifying backends that do not support two-argument functions `f!(y, x)`. +Trait identifying backends that do not support in-place functions `f!(y, x)`. """ -struct TwoArgNotSupported <: MutationBehavior end +struct InPlaceNotSupported <: InPlaceBehavior end """ - twoarg_support(backend) + inplace_support(backend) -Return [`TwoArgSupported`](@ref) or [`TwoArgNotSupported`](@ref) in a statically predictable way. +Return [`InPlaceSupported`](@ref) or [`InPlaceNotSupported`](@ref) in a statically predictable way. """ -twoarg_support(::AbstractADType) = TwoArgSupported() +inplace_support(::AbstractADType) = InPlaceSupported() -function twoarg_support(backend::SecondOrder) - if Bool(twoarg_support(inner(backend))) && Bool(twoarg_support(outer(backend))) - return TwoArgSupported() +function inplace_support(backend::SecondOrder) + if Bool(inplace_support(inner(backend))) && Bool(inplace_support(outer(backend))) + return InPlaceSupported() else - return TwoArgNotSupported() + return InPlaceNotSupported() end end -twoarg_support(backend::AutoSparse) = twoarg_support(dense_ad(backend)) +inplace_support(backend::AutoSparse) = inplace_support(dense_ad(backend)) ## Pushforward @@ -142,8 +142,8 @@ hvp_mode(backend::AutoSparse{<:SecondOrder}) = hvp_mode(dense_ad(backend)) ## Conversions -Base.Bool(::TwoArgSupported) = true -Base.Bool(::TwoArgNotSupported) = false +Base.Bool(::InPlaceSupported) = true +Base.Bool(::InPlaceNotSupported) = false Base.Bool(::PushforwardFast) = true Base.Bool(::PushforwardSlow) = false diff --git a/DifferentiationInterface/test/Back/ChainRulesCore/test.jl b/DifferentiationInterface/test/Back/ChainRulesCore/test.jl index 31388380c..149c428e5 100644 --- a/DifferentiationInterface/test/Back/ChainRulesCore/test.jl +++ b/DifferentiationInterface/test/Back/ChainRulesCore/test.jl @@ -10,8 +10,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoChainRules(ZygoteRuleConfig())] @test check_available(backend) - @test !check_twoarg(backend) - @test check_hessian(backend) + @test !check_inplace(backend) end test_differentiation( diff --git a/DifferentiationInterface/test/Back/Diffractor/test.jl b/DifferentiationInterface/test/Back/Diffractor/test.jl index c50f244a0..23f6412a6 100644 --- a/DifferentiationInterface/test/Back/Diffractor/test.jl +++ b/DifferentiationInterface/test/Back/Diffractor/test.jl @@ -9,8 +9,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoDiffractor()] @test check_available(backend) - @test !check_twoarg(backend) - @test !check_hessian(backend; verbose=false) + @test !check_inplace(backend) end test_differentiation( diff --git a/DifferentiationInterface/test/Back/Enzyme/test.jl b/DifferentiationInterface/test/Back/Enzyme/test.jl index 1c6b82b9b..bf8ba2574 100644 --- a/DifferentiationInterface/test/Back/Enzyme/test.jl +++ b/DifferentiationInterface/test/Back/Enzyme/test.jl @@ -40,8 +40,7 @@ sparse_backends = @testset "Checks" begin @testset "Check $(typeof(backend))" for backend in vcat(dense_backends, sparse_backends) @test check_available(backend) - @test check_twoarg(backend) - @test check_hessian(backend; verbose=false) + @test check_inplace(backend) end end diff --git a/DifferentiationInterface/test/Back/FastDifferentiation/test.jl b/DifferentiationInterface/test/Back/FastDifferentiation/test.jl index 763d0de0c..ff297f7f4 100644 --- a/DifferentiationInterface/test/Back/FastDifferentiation/test.jl +++ b/DifferentiationInterface/test/Back/FastDifferentiation/test.jl @@ -9,8 +9,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoFastDifferentiation(), AutoSparse(AutoFastDifferentiation())] @test check_available(backend) - @test check_twoarg(backend) - @test check_hessian(backend) + @test check_inplace(backend) end test_differentiation(AutoFastDifferentiation(); logging=LOGGING); diff --git a/DifferentiationInterface/test/Back/FiniteDiff/test.jl b/DifferentiationInterface/test/Back/FiniteDiff/test.jl index c69bfe403..8246ebbf3 100644 --- a/DifferentiationInterface/test/Back/FiniteDiff/test.jl +++ b/DifferentiationInterface/test/Back/FiniteDiff/test.jl @@ -9,8 +9,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoFiniteDiff()] @test check_available(backend) - @test check_twoarg(backend) - @test check_hessian(backend) + @test check_inplace(backend) end test_differentiation(AutoFiniteDiff(); excluded=[:second_derivative, :hvp], logging=LOGGING); diff --git a/DifferentiationInterface/test/Back/FiniteDifferences/test.jl b/DifferentiationInterface/test/Back/FiniteDifferences/test.jl index 326f6a48b..1b0b42696 100644 --- a/DifferentiationInterface/test/Back/FiniteDifferences/test.jl +++ b/DifferentiationInterface/test/Back/FiniteDifferences/test.jl @@ -9,8 +9,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoFiniteDifferences(; fdm=FiniteDifferences.central_fdm(3, 1))] @test check_available(backend) - @test !check_twoarg(backend) - @test_broken !check_hessian(backend; verbose=false) + @test !check_inplace(backend) end test_differentiation( diff --git a/DifferentiationInterface/test/Back/ForwardDiff/test.jl b/DifferentiationInterface/test/Back/ForwardDiff/test.jl index a023ba8f9..ea87c5300 100644 --- a/DifferentiationInterface/test/Back/ForwardDiff/test.jl +++ b/DifferentiationInterface/test/Back/ForwardDiff/test.jl @@ -22,8 +22,7 @@ sparse_backends = [ for backend in vcat(dense_backends, sparse_backends) @test check_available(backend) - @test check_twoarg(backend) - @test check_hessian(backend) + @test check_inplace(backend) end ## Dense backends diff --git a/DifferentiationInterface/test/Back/PolyesterForwardDiff/test.jl b/DifferentiationInterface/test/Back/PolyesterForwardDiff/test.jl index dc56d3820..5cfdf96ab 100644 --- a/DifferentiationInterface/test/Back/PolyesterForwardDiff/test.jl +++ b/DifferentiationInterface/test/Back/PolyesterForwardDiff/test.jl @@ -9,8 +9,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoPolyesterForwardDiff(; chunksize=1)] @test check_available(backend) - @test check_twoarg(backend) - @test check_hessian(backend) + @test check_inplace(backend) end test_differentiation(AutoPolyesterForwardDiff(; chunksize=1); logging=LOGGING); diff --git a/DifferentiationInterface/test/Back/ReverseDiff/test.jl b/DifferentiationInterface/test/Back/ReverseDiff/test.jl index b8d1707f1..c2144562e 100644 --- a/DifferentiationInterface/test/Back/ReverseDiff/test.jl +++ b/DifferentiationInterface/test/Back/ReverseDiff/test.jl @@ -15,8 +15,7 @@ fromprimitive_backends = [AutoReverseFromPrimitive(AutoReverseDiff())] for backend in vcat(dense_backends, fromprimitive_backends) @test check_available(backend) - @test check_twoarg(backend) - @test check_hessian(backend) + @test check_inplace(backend) end test_differentiation(vcat(dense_backends, fromprimitive_backends); logging=LOGGING); diff --git a/DifferentiationInterface/test/Back/SecondOrder/test.jl b/DifferentiationInterface/test/Back/SecondOrder/test.jl index 04534f053..e92370200 100644 --- a/DifferentiationInterface/test/Back/SecondOrder/test.jl +++ b/DifferentiationInterface/test/Back/SecondOrder/test.jl @@ -33,11 +33,10 @@ twoarg_backends = [ for backend in vcat(onearg_backends, twoarg_backends) @test check_available(backend) if backend in onearg_backends - @test !check_twoarg(backend) + @test !check_inplace(backend) else - @test check_twoarg(backend) + @test check_inplace(backend) end - @test check_hessian(backend) end test_differentiation( diff --git a/DifferentiationInterface/test/Back/Symbolics/test.jl b/DifferentiationInterface/test/Back/Symbolics/test.jl index 26a13dc99..33ce06f95 100644 --- a/DifferentiationInterface/test/Back/Symbolics/test.jl +++ b/DifferentiationInterface/test/Back/Symbolics/test.jl @@ -9,8 +9,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoSymbolics(), AutoSparse(AutoSymbolics())] @test check_available(backend) - @test check_twoarg(backend) - @test check_hessian(backend) + @test check_inplace(backend) end test_differentiation(AutoSymbolics(); logging=LOGGING); diff --git a/DifferentiationInterface/test/Back/Tapir/test.jl b/DifferentiationInterface/test/Back/Tapir/test.jl index 697b14743..058aed793 100644 --- a/DifferentiationInterface/test/Back/Tapir/test.jl +++ b/DifferentiationInterface/test/Back/Tapir/test.jl @@ -9,8 +9,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoTapir(; safe_mode=false)] @test check_available(backend) - @test check_twoarg(backend) - @test !check_hessian(backend; verbose=false) + @test check_inplace(backend) end # Safe mode switched off to avoid polluting the test suite with diff --git a/DifferentiationInterface/test/Back/Tracker/test.jl b/DifferentiationInterface/test/Back/Tracker/test.jl index 04efcf8a1..e22d01a01 100644 --- a/DifferentiationInterface/test/Back/Tracker/test.jl +++ b/DifferentiationInterface/test/Back/Tracker/test.jl @@ -9,8 +9,7 @@ LOGGING = get(ENV, "CI", "false") == "false" for backend in [AutoTracker()] @test check_available(backend) - @test !check_twoarg(backend) - @test !check_hessian(backend; verbose=false) + @test !check_inplace(backend) end test_differentiation(AutoTracker(); second_order=false, logging=LOGGING); diff --git a/DifferentiationInterface/test/Back/Zygote/test.jl b/DifferentiationInterface/test/Back/Zygote/test.jl index 017ba2efc..13b38500f 100644 --- a/DifferentiationInterface/test/Back/Zygote/test.jl +++ b/DifferentiationInterface/test/Back/Zygote/test.jl @@ -23,8 +23,7 @@ sparse_backends = [ for backend in vcat(dense_backends, sparse_backends) @test check_available(backend) - @test !check_twoarg(backend) - @test check_hessian(backend) + @test !check_inplace(backend) end ## Dense backends diff --git a/DifferentiationInterface/test/Internals/backends.jl b/DifferentiationInterface/test/Internals/backends.jl index 9d6a4bfba..22bd97bf5 100644 --- a/DifferentiationInterface/test/Internals/backends.jl +++ b/DifferentiationInterface/test/Internals/backends.jl @@ -15,7 +15,7 @@ end sparse_backend = AutoSparse(backend) @test ADTypes.mode(sparse_backend) == ADTypes.mode(backend) @test check_available(sparse_backend) == check_available(backend) - @test DI.twoarg_support(sparse_backend) == DI.twoarg_support(backend) + @test DI.inplace_support(sparse_backend) == DI.inplace_support(backend) @test DI.pushforward_performance(sparse_backend) == DI.pushforward_performance(backend) @test DI.pullback_performance(sparse_backend) == DI.pullback_performance(backend) diff --git a/DifferentiationInterface/test/Internals/from_primitive.jl b/DifferentiationInterface/test/Internals/from_primitive.jl index 7ef9c2884..06067b535 100644 --- a/DifferentiationInterface/test/Internals/from_primitive.jl +++ b/DifferentiationInterface/test/Internals/from_primitive.jl @@ -12,8 +12,7 @@ fromprimitive_backends = [ # for backend in vcat(fromprimitive_backends) @test check_available(backend) - @test check_twoarg(backend) - @test check_hessian(backend) + @test check_inplace(backend) @test DifferentiationInterface.pick_batchsize(backend, 100) == 5 end diff --git a/DifferentiationInterface/test/Internals/zero_backends.jl b/DifferentiationInterface/test/Internals/zero_backends.jl index a73348f97..fb48580c6 100644 --- a/DifferentiationInterface/test/Internals/zero_backends.jl +++ b/DifferentiationInterface/test/Internals/zero_backends.jl @@ -12,7 +12,7 @@ zero_backends = [AutoZeroForward(), AutoZeroReverse()] for backend in zero_backends @test check_available(backend) - @test check_twoarg(backend) + @test check_inplace(backend) end ## Type stability diff --git a/DifferentiationInterfaceTest/src/DifferentiationInterfaceTest.jl b/DifferentiationInterfaceTest/src/DifferentiationInterfaceTest.jl index 3e43a47c5..d96c0c5c5 100644 --- a/DifferentiationInterfaceTest/src/DifferentiationInterfaceTest.jl +++ b/DifferentiationInterfaceTest/src/DifferentiationInterfaceTest.jl @@ -28,7 +28,7 @@ using DifferentiationInterface: maybe_dense_ad, mode, outer, - twoarg_support, + inplace_support, pushforward_performance, pullback_performance using DifferentiationInterface: diff --git a/DifferentiationInterfaceTest/src/scenarios/scenario.jl b/DifferentiationInterfaceTest/src/scenarios/scenario.jl index ccaaf60b3..ffc124490 100644 --- a/DifferentiationInterfaceTest/src/scenarios/scenario.jl +++ b/DifferentiationInterfaceTest/src/scenarios/scenario.jl @@ -75,7 +75,7 @@ end function compatible(backend::AbstractADType, scen::Scenario) if nb_args(scen) == 2 - return Bool(twoarg_support(backend)) + return Bool(inplace_support(backend)) end return true end diff --git a/DifferentiationInterfaceTest/src/test_differentiation.jl b/DifferentiationInterfaceTest/src/test_differentiation.jl index 91b4c42ad..f41a4493d 100644 --- a/DifferentiationInterfaceTest/src/test_differentiation.jl +++ b/DifferentiationInterfaceTest/src/test_differentiation.jl @@ -20,7 +20,7 @@ Filtering: - `input_type=Any`, `output_type=Any`: restrict scenario inputs / outputs to subtypes of this - `first_order=true`, `second_order=true`: include first order / second order operators -- `onearg=true`, `twoarg=true`: include one-argument / two-argument functions +- `onearg=true`, `twoarg=true`: include out-of-place / in-place functions - `inplace=true`, `outofplace=true`: include in-place / out-of-place operators Options: