diff --git a/dev/api/index.html b/dev/api/index.html index 327317873c..f7b148744a 100644 --- a/dev/api/index.html +++ b/dev/api/index.html @@ -1,5 +1,5 @@ -API · Enzyme.jl
Edit on GitHub

API

Types and constants

Functions and macros

Documentation

Enzyme.gradient! — Method
gradient!(::ReverseMode, dx, f, x)

Compute the gradient of an array-input function f using reverse mode, storing the derivative result in an existing array dx.

Example:

f(x) = x[1]*x[2]
+API · Enzyme.jl
Edit on GitHub
API
Types and constants
Functions and macros
Documentation
Enzyme.gradient! — Method
gradient!(::ReverseMode, dx, f, x)
Compute the gradient of an array-input function f using reverse mode, storing the derivative result in an existing array dx.
Example:
f(x) = x[1]*x[2]
 
 dx = [0.0, 0.0]
 gradient!(Reverse, dx, f, [2.0, 3.0])
@@ -8,13 +8,13 @@
 
 2-element Vector{Float64}:
  3.0
- 2.0
source
Enzyme.gradient — Method
gradient(::ForwardMode, f, x; shadow=onehot(x))
Compute the gradient of an array-input function f using forward mode. The optional keyword argument shadow is a vector of one-hot vectors of type x which are used to forward-propagate into the return. For performance reasons, this should be computed once, outside the call to gradient, rather than within this call.
Example:
f(x) = x[1]*x[2]
+ 2.0
source
Enzyme.gradient — Method
gradient(::ForwardMode, f, x; shadow=onehot(x))
Compute the gradient of an array-input function f using forward mode. The optional keyword argument shadow is a vector of one-hot vectors of type x which are used to forward-propagate into the return. For performance reasons, this should be computed once, outside the call to gradient, rather than within this call.
Example:
f(x) = x[1]*x[2]
 
 grad = gradient(Forward, f, [2.0, 3.0])
 
 # output
 
-(3.0, 2.0)
source
Enzyme.gradient — Method
gradient(::ReverseMode, f, x)
Compute the gradient of an array-input function f using reverse mode. This will allocate and return new array with the gradient result.
Example:
f(x) = x[1]*x[2]
+(3.0, 2.0)
source
Enzyme.gradient — Method
gradient(::ReverseMode, f, x)
Compute the gradient of an array-input function f using reverse mode. This will allocate and return new array with the gradient result.
Example:
f(x) = x[1]*x[2]
 
 grad = gradient(Reverse, f, [2.0, 3.0])
 
@@ -22,13 +22,13 @@
 
 2-element Vector{Float64}:
  3.0
- 2.0
source
Enzyme.gradient — Method
gradient(::ForwardMode, f, x, ::Val{chunk}; shadow=onehot(x))
Compute the gradient of an array-input function f using vector forward mode. Like gradient, except it uses a chunk size of chunk to compute chunk derivatives in a single call.
Example:
f(x) = x[1]*x[2]
+ 2.0
source
Enzyme.gradient — Method
gradient(::ForwardMode, f, x, ::Val{chunk}; shadow=onehot(x))
Compute the gradient of an array-input function f using vector forward mode. Like gradient, except it uses a chunk size of chunk to compute chunk derivatives in a single call.
Example:
f(x) = x[1]*x[2]
 
 grad = gradient(Forward, f, [2.0, 3.0], Val(2))
 
 # output
 
-(3.0, 2.0)
source
Enzyme.jacobian — Method
jacobian(::ForwardMode, f, x; shadow=onehot(x))
 jacobian(::ForwardMode, f, x, ::Val{chunk}; shadow=onehot(x))
Compute the jacobian of an array-input function f using (potentially vector) forward mode. This is a simple rename of the gradient function, and all relevant arguments apply here.
Example:
f(x) = [x[1]*x[2], x[2]]
 
 grad = jacobian(Forward, f, [2.0, 3.0])
@@ -37,7 +37,7 @@
 
 2×2 Matrix{Float64}:
  3.0  2.0
- 0.0  1.0
source
Enzyme.jacobian — Method
jacobian(::ReverseMode, f, x, ::Val{num_outs}, ::Val{chunk})
Compute the jacobian of an array-input function f using (potentially vector) reverse mode. The chunk argument denotes the chunk size to use and num_outs denotes the number of outputs f will return in an array.
Example:
f(x) = [x[1]*x[2], x[2]]
+ 0.0  1.0
source
Enzyme.jacobian — Method
jacobian(::ReverseMode, f, x, ::Val{num_outs}, ::Val{chunk})
Compute the jacobian of an array-input function f using (potentially vector) reverse mode. The chunk argument denotes the chunk size to use and num_outs denotes the number of outputs f will return in an array.
Example:
f(x) = [x[1]*x[2], x[2]]
 
 grad = jacobian(Reverse, f, [2.0, 3.0], Val(2))
 
@@ -45,7 +45,7 @@
 
 2×2 Matrix{Float64}:
  3.0  2.0
- 0.0  1.0
source
EnzymeCore.autodiff — Method
autodiff(::ForwardMode, f, Activity, args::Vararg{Annotation, Nargs})
Auto-differentiate function f at arguments args using forward mode.
args may be numbers, arrays, structs of numbers, structs of arrays and so on. Enzyme will only differentiate in respect to arguments that are wrapped in a Duplicated or similar argument. Unlike reverse mode in autodiff, Active arguments are not allowed here, since all derivative results of immutable objects will be returned and should instead use Duplicated or variants like DuplicatedNoNeed.
Activity is the Activity of the return value, it may be:
  • Const if the return is not to be differentiated with respect to
  • Duplicated, if the return is being differentiated with respect to and both the original value and the derivative return are desired
  • DuplicatedNoNeed, if the return is being differentiated with respect to and only the derivative return is desired.
  • BatchDuplicated, like Duplicated, but computing multiple derivatives at once. All batch sizes must be the same for all arguments.
  • BatchDuplicatedNoNeed, like DuplicatedNoNeed, but computing multiple derivatives at one. All batch sizes must be the same for all arguments.
Example returning both original return and derivative:
f(x) = x*x
+ 0.0  1.0
source
EnzymeCore.autodiff — Method
autodiff(::ForwardMode, f, Activity, args::Vararg{Annotation, Nargs})
Auto-differentiate function f at arguments args using forward mode.
args may be numbers, arrays, structs of numbers, structs of arrays and so on. Enzyme will only differentiate in respect to arguments that are wrapped in a Duplicated or similar argument. Unlike reverse mode in autodiff, Active arguments are not allowed here, since all derivative results of immutable objects will be returned and should instead use Duplicated or variants like DuplicatedNoNeed.
Activity is the Activity of the return value, it may be:
  • Const if the return is not to be differentiated with respect to
  • Duplicated, if the return is being differentiated with respect to and both the original value and the derivative return are desired
  • DuplicatedNoNeed, if the return is being differentiated with respect to and only the derivative return is desired.
  • BatchDuplicated, like Duplicated, but computing multiple derivatives at once. All batch sizes must be the same for all arguments.
  • BatchDuplicatedNoNeed, like DuplicatedNoNeed, but computing multiple derivatives at one. All batch sizes must be the same for all arguments.
Example returning both original return and derivative:
f(x) = x*x
 res, ∂f_∂x = autodiff(Forward, f, Duplicated, Duplicated(3.14, 1.0))
 
 # output
@@ -55,7 +55,7 @@
 
 # output
 
-(6.28,)
source
EnzymeCore.autodiff — Method
autodiff(::ReverseMode, f, Activity, args::Vararg{Annotation, Nargs})
Auto-differentiate function f at arguments args using reverse mode.
Limitations:
  • f may only return a Real (of a built-in/primitive type) or nothing, not an array, struct, BigFloat, etc. To handle vector-valued return types, use a mutating f! that returns nothing and stores it's return value in one of the arguments, which must be wrapped in a Duplicated.
args may be numbers, arrays, structs of numbers, structs of arrays and so on. Enzyme will only differentiate in respect to arguments that are wrapped in an Active (for arguments whose derivative result must be returned rather than mutated in place, such as primitive types and structs thereof) or Duplicated (for mutable arguments like arrays, Refs and structs thereof).
Activity is the Activity of the return value, it may be Const or Active.
Example:
a = 4.2
+(6.28,)
source
EnzymeCore.autodiff — Method
autodiff(::ReverseMode, f, Activity, args::Vararg{Annotation, Nargs})
Auto-differentiate function f at arguments args using reverse mode.
Limitations:
  • f may only return a Real (of a built-in/primitive type) or nothing, not an array, struct, BigFloat, etc. To handle vector-valued return types, use a mutating f! that returns nothing and stores it's return value in one of the arguments, which must be wrapped in a Duplicated.
args may be numbers, arrays, structs of numbers, structs of arrays and so on. Enzyme will only differentiate in respect to arguments that are wrapped in an Active (for arguments whose derivative result must be returned rather than mutated in place, such as primitive types and structs thereof) or Duplicated (for mutable arguments like arrays, Refs and structs thereof).
Activity is the Activity of the return value, it may be Const or Active.
Example:
a = 4.2
 b = [2.2, 3.3]; ∂f_∂b = zero(b)
 c = 55; d = 9
 
@@ -68,7 +68,7 @@
 
 # output
 
-((6.0,), 9.0)
Note
Enzyme gradients with respect to integer values are zero. Active will automatically convert plain integers to floating point values, but cannot do so for integer values in tuples and structs.
source
EnzymeCore.autodiff — Method
autodiff(mode::Mode, f, ::Type{A}, args::Vararg{Annotation, Nargs})
Like autodiff but will try to extend f to an annotation, if needed.
source
EnzymeCore.autodiff_deferred — Method
autodiff_deferred(::ForwardMode, f, Activity, args::Vararg{Annotation, Nargs})
Same as autodiff(::ForwardMode, f, Activity, args) but uses deferred compilation to support usage in GPU code, as well as high-order differentiation.
source
EnzymeCore.autodiff_deferred — Method
autodiff_deferred(::ReverseMode, f, Activity, args::Vararg{Annotation, Nargs})
Same as autodiff but uses deferred compilation to support usage in GPU code, as well as high-order differentiation.
source
EnzymeCore.autodiff_deferred_thunk — Method
autodiff_deferred_thunk(::ReverseModeSplit, ftype, Activity, argtypes::Vararg{Type{<:Annotation}, Nargs})
Provide the split forward and reverse pass functions for annotated function type ftype when called with args of type argtypes when using reverse mode.
Activity is the Activity of the return value, it may be Const, Active, or Duplicated (or its variants DuplicatedNoNeed, BatchDuplicated, and BatchDuplicatedNoNeed).
The forward function will return a tape, the primal (or nothing if not requested), and the shadow (or nothing if not a Duplicated variant), and tapes the corresponding type arguements provided.
The reverse function will return the derivative of Active arguments, updating the Duplicated arguments in place. The same arguments to the forward pass should be provided, followed by the adjoint of the return (if the return is active), and finally the tape from the forward pass.
Example:

+((6.0,), 9.0)
Note
Enzyme gradients with respect to integer values are zero. Active will automatically convert plain integers to floating point values, but cannot do so for integer values in tuples and structs.
source
EnzymeCore.autodiff — Method
autodiff(mode::Mode, f, ::Type{A}, args::Vararg{Annotation, Nargs})
Like autodiff but will try to extend f to an annotation, if needed.
source
EnzymeCore.autodiff_deferred — Method
autodiff_deferred(::ForwardMode, f, Activity, args::Vararg{Annotation, Nargs})
Same as autodiff(::ForwardMode, f, Activity, args) but uses deferred compilation to support usage in GPU code, as well as high-order differentiation.
source
EnzymeCore.autodiff_deferred — Method
autodiff_deferred(::ReverseMode, f, Activity, args::Vararg{Annotation, Nargs})
Same as autodiff but uses deferred compilation to support usage in GPU code, as well as high-order differentiation.
source
EnzymeCore.autodiff_deferred_thunk — Method
autodiff_deferred_thunk(::ReverseModeSplit, ftype, Activity, argtypes::Vararg{Type{<:Annotation}, Nargs})
Provide the split forward and reverse pass functions for annotated function type ftype when called with args of type argtypes when using reverse mode.
Activity is the Activity of the return value, it may be Const, Active, or Duplicated (or its variants DuplicatedNoNeed, BatchDuplicated, and BatchDuplicatedNoNeed).
The forward function will return a tape, the primal (or nothing if not requested), and the shadow (or nothing if not a Duplicated variant), and tapes the corresponding type arguements provided.
The reverse function will return the derivative of Active arguments, updating the Duplicated arguments in place. The same arguments to the forward pass should be provided, followed by the adjoint of the return (if the return is active), and finally the tape from the forward pass.
Example:

 A = [2.2]; ∂A = zero(A)
 v = 3.3
 
@@ -88,7 +88,7 @@
 
 # output
 
-(7.26, 2.2, [3.3])
source
EnzymeCore.autodiff_thunk — Method
autodiff_thunk(::ForwardMode, ftype, Activity, argtypes::Vararg{Type{<:Annotation}, Nargs})
Provide the thunk forward mode function for annotated function type ftype when called with args of type argtypes.
Activity is the Activity of the return value, it may be Const or Duplicated (or its variants DuplicatedNoNeed, BatchDuplicated, andBatchDuplicatedNoNeed).
The forward function will return the primal (if requested) and the shadow (or nothing if not a Duplicated variant).
Example returning both original return and derivative:
a = 4.2
+(7.26, 2.2, [3.3])
source
EnzymeCore.autodiff_thunk — Method
autodiff_thunk(::ForwardMode, ftype, Activity, argtypes::Vararg{Type{<:Annotation}, Nargs})
Provide the thunk forward mode function for annotated function type ftype when called with args of type argtypes.
Activity is the Activity of the return value, it may be Const or Duplicated (or its variants DuplicatedNoNeed, BatchDuplicated, andBatchDuplicatedNoNeed).
The forward function will return the primal (if requested) and the shadow (or nothing if not a Duplicated variant).
Example returning both original return and derivative:
a = 4.2
 b = [2.2, 3.3]; ∂f_∂b = zero(b)
 c = 55; d = 9
 
@@ -108,7 +108,7 @@
 
 # output
 
-(6.28,)
source
EnzymeCore.autodiff_thunk — Method
autodiff_thunk(::ReverseModeSplit, ftype, Activity, argtypes::Vararg{Type{<:Annotation}, Nargs})
Provide the split forward and reverse pass functions for annotated function type ftype when called with args of type argtypes when using reverse mode.
Activity is the Activity of the return value, it may be Const, Active, or Duplicated (or its variants DuplicatedNoNeed, BatchDuplicated, and BatchDuplicatedNoNeed).
The forward function will return a tape, the primal (or nothing if not requested), and the shadow (or nothing if not a Duplicated variant), and tapes the corresponding type arguements provided.
The reverse function will return the derivative of Active arguments, updating the Duplicated arguments in place. The same arguments to the forward pass should be provided, followed by the adjoint of the return (if the return is active), and finally the tape from the forward pass.
Example:

+(6.28,)
source
EnzymeCore.autodiff_thunk — Method
autodiff_thunk(::ReverseModeSplit, ftype, Activity, argtypes::Vararg{Type{<:Annotation}, Nargs})
Provide the split forward and reverse pass functions for annotated function type ftype when called with args of type argtypes when using reverse mode.
Activity is the Activity of the return value, it may be Const, Active, or Duplicated (or its variants DuplicatedNoNeed, BatchDuplicated, and BatchDuplicatedNoNeed).
The forward function will return a tape, the primal (or nothing if not requested), and the shadow (or nothing if not a Duplicated variant), and tapes the corresponding type arguements provided.
The reverse function will return the derivative of Active arguments, updating the Duplicated arguments in place. The same arguments to the forward pass should be provided, followed by the adjoint of the return (if the return is active), and finally the tape from the forward pass.
Example:

 A = [2.2]; ∂A = zero(A)
 v = 3.3
 
@@ -127,17 +127,17 @@
 
 # output
 
-(7.26, 2.2, [3.3])
source
EnzymeCore.Active — Type
Active(x)
Mark a function argument x of autodiff as active, Enzyme will auto-differentiate in respect Active arguments.
Note
Enzyme gradients with respect to integer values are zero. Active will automatically convert plain integers to floating point values, but cannot do so for integer values in tuples and structs.
source
EnzymeCore.BatchDuplicated — Type
BatchDuplicated(x, ∂f_∂xs)
Like Duplicated, except contains several shadows to compute derivatives for all at once. Argument ∂f_∂xs should be a tuple of the several values of type x.
source
EnzymeCore.Const — Type
Const(x)
Mark a function argument x of autodiff as constant, Enzyme will not auto-differentiate in respect Const arguments.
source
EnzymeCore.Duplicated — Type
Duplicated(x, ∂f_∂x)
Mark a function argument x of autodiff as duplicated, Enzyme will auto-differentiate in respect to such arguments, with dx acting as an accumulator for gradients (so $\partial f / \partial x$ will be added to) ∂f_∂x.
source
EnzymeCore.FFIABI — Type
struct FFIABI <: ABI
Foreign function call ABI. JIT the differentiated function, then inttoptr call the address.
source
EnzymeCore.Mode — Type
abstract type Mode
Abstract type for what differentiation mode will be used.
source
EnzymeCore.ReverseMode — Type
struct ReverseMode{ReturnPrimal,ABI,Holomorphic} <: Mode{ABI}
Reverse mode differentiation.
  • ReturnPrimal: Should Enzyme return the primal return value from the augmented-forward.
  • ABI: What runtime ABI to use
  • Holomorphic: Whether the complex result function is holomorphic and we should compute d/dz
source
EnzymeCore.ReverseModeSplit — Type
struct ReverseModeSplit{ReturnPrimal,ReturnShadow,Width,ModifiedBetween,ABI} <: Mode{ABI}
Reverse mode differentiation.
  • ReturnPrimal: Should Enzyme return the primal return value from the augmented-forward.
  • ReturnShadow: Should Enzyme return the shadow return value from the augmented-forward.
  • Width: Batch Size (0 if to be automatically derived)
  • ModifiedBetween: Tuple of each argument's modified between state (true if to be automatically derived).
source
EnzymeCore.compiler_job_from_backend — Function
compiler_job_from_backend(::KernelAbstractions.Backend, F::Type, TT:Type)::GPUCompiler.CompilerJob
Returns a GPUCompiler CompilerJob from a backend as specified by the first argument to the function.
For example, in CUDA one would do:
function EnzymeCore.compiler_job_from_backend(::CUDABackend, @nospecialize(F::Type), @nospecialize(TT::Type))
+(7.26, 2.2, [3.3])
source
EnzymeCore.Active — Type
Active(x)
Mark a function argument x of autodiff as active, Enzyme will auto-differentiate in respect Active arguments.
Note
Enzyme gradients with respect to integer values are zero. Active will automatically convert plain integers to floating point values, but cannot do so for integer values in tuples and structs.
source
EnzymeCore.BatchDuplicated — Type
BatchDuplicated(x, ∂f_∂xs)
Like Duplicated, except contains several shadows to compute derivatives for all at once. Argument ∂f_∂xs should be a tuple of the several values of type x.
source
EnzymeCore.Const — Type
Const(x)
Mark a function argument x of autodiff as constant, Enzyme will not auto-differentiate in respect Const arguments.
source
EnzymeCore.Duplicated — Type
Duplicated(x, ∂f_∂x)
Mark a function argument x of autodiff as duplicated, Enzyme will auto-differentiate in respect to such arguments, with dx acting as an accumulator for gradients (so $\partial f / \partial x$ will be added to) ∂f_∂x.
source
EnzymeCore.FFIABI — Type
struct FFIABI <: ABI
Foreign function call ABI. JIT the differentiated function, then inttoptr call the address.
source
EnzymeCore.Mode — Type
abstract type Mode
Abstract type for what differentiation mode will be used.
source
EnzymeCore.ReverseMode — Type
struct ReverseMode{ReturnPrimal,ABI,Holomorphic} <: Mode{ABI}
Reverse mode differentiation.
  • ReturnPrimal: Should Enzyme return the primal return value from the augmented-forward.
  • ABI: What runtime ABI to use
  • Holomorphic: Whether the complex result function is holomorphic and we should compute d/dz
source
EnzymeCore.ReverseModeSplit — Type
struct ReverseModeSplit{ReturnPrimal,ReturnShadow,Width,ModifiedBetween,ABI} <: Mode{ABI}
Reverse mode differentiation.
  • ReturnPrimal: Should Enzyme return the primal return value from the augmented-forward.
  • ReturnShadow: Should Enzyme return the shadow return value from the augmented-forward.
  • Width: Batch Size (0 if to be automatically derived)
  • ModifiedBetween: Tuple of each argument's modified between state (true if to be automatically derived).
source
EnzymeCore.compiler_job_from_backend — Function
compiler_job_from_backend(::KernelAbstractions.Backend, F::Type, TT:Type)::GPUCompiler.CompilerJob
Returns a GPUCompiler CompilerJob from a backend as specified by the first argument to the function.
For example, in CUDA one would do:
function EnzymeCore.compiler_job_from_backend(::CUDABackend, @nospecialize(F::Type), @nospecialize(TT::Type))
     mi = GPUCompiler.methodinstance(F, TT)
     return GPUCompiler.CompilerJob(mi, CUDA.compiler_config(CUDA.device()))
-end
source
EnzymeCore.make_zero — Function
make_zero(::Type{T}, seen::IdDict, prev::T, ::Val{copy_if_inactive}=Val(false))::T
 
 Recursively make a zero'd copy of the value `prev` of type `T`. The argument `copy_if_inactive` specifies
-what to do if the type `T` is guaranteed to be inactive, use the primal (the default) or still copy the value.
source
EnzymeCore.make_zero — Method
make_zero(prev::T)
+what to do if the type `T` is guaranteed to be inactive, use the primal (the default) or still copy the value.
source
EnzymeCore.EnzymeRules.AugmentedReturn — Type
AugmentedReturn(primal, shadow, tape)
Augment the primal return value of a function with its shadow, as well as any additional information needed to correctly  compute the reverse pass, stored in tape.
Unless specified by the config that a variable is not overwritten, rules must assume any arrays/data structures/etc are  overwritten between the forward and the reverse pass. Any floats or variables passed by value are always preserved as is  (as are the arrays themselves, just not necessarily the values in the array).
See also augmented_primal.
source
EnzymeCore.EnzymeRules.Config — Type
Config{NeedsPrimal, NeedsShadow, Width, Overwritten}
-ConfigWidth{Width} = Config{<:Any,<:Any, Width}
Configuration type to dispatch on in custom reverse rules (see augmented_primal and reverse).
  • NeedsPrimal and NeedsShadow: boolean values specifying whether the primal and shadow (resp.) should be returned. 
  • Width: an integer that specifies the number of adjoints/shadows simultaneously being propagated.
  • Overwritten: a tuple of booleans of whether each argument (including the function itself) is modified between the   forward and reverse pass (true if potentially modified between).
Getters for the four type parameters are provided by needs_primal, needs_shadow, width, and overwritten.
source
EnzymeCore.EnzymeRules.augmented_primal — Function
augmented_primal(::Config, func::Annotation{typeof(f)}, RT::Type{<:Annotation}, args::Annotation...)
Must return an AugmentedReturn type.
  • The primal must be the same type of the original return if needs_primal(config), otherwise nothing.
  • The shadow must be nothing if needs_shadow(config) is false. If width is 1, the shadow should be the same type of the original return. If the width is greater than 1, the shadow should be NTuple{original return, width}.
  • The tape can be any type (including Nothing) and is preserved for the reverse call.
source
EnzymeCore.EnzymeRules.forward — Function
forward(func::Annotation{typeof(f)}, RT::Type{<:Annotation}, args::Annotation...)
Calculate the forward derivative. The first argument func is the callable for which the rule applies to. Either wrapped in a Const), or a Duplicated if it is a closure. The second argument is the return type annotation, and all other arguments are the annotated function arguments.
source
EnzymeCore.EnzymeRules.inactive — Function
inactive(func::typeof(f), args...)
Mark a particular function as always being inactive in both its return result and the function call itself.
source
EnzymeCore.EnzymeRules.inactive_noinl — Function
inactive_noinl(func::typeof(f), args...)
Mark a particular function as always being inactive in both its return result and the function call itself, but do not prevent inlining of the function.
source
EnzymeCore.EnzymeRules.reverse — Function
reverse(::Config, func::Annotation{typeof(f)}, dret::Active, tape, args::Annotation...)
-reverse(::Config, func::Annotation{typeof(f)}, ::Type{<:Annotation), tape, args::Annotation...)
Takes gradient of derivative, activity annotation, and tape. If there is an active return dret is passed as Active{T} with the derivative of the active return val. Otherwise dret is passed as Type{Duplicated{T}}, etc.
source
EnzymeTestUtils.@test_msg — Macro
@test_msg msg condion kws...
This is per Test.@test condion kws... except that if it fails it also prints the msg. If msg=="" then this is just like @test, nothing is printed
Examles
julia> @test_msg "It is required that the total is under 10" sum(1:1000) < 10;
+Helper function to recursively make zero.
source
EnzymeCore.EnzymeRules.AugmentedReturn — Type
AugmentedReturn(primal, shadow, tape)
Augment the primal return value of a function with its shadow, as well as any additional information needed to correctly  compute the reverse pass, stored in tape.
Unless specified by the config that a variable is not overwritten, rules must assume any arrays/data structures/etc are  overwritten between the forward and the reverse pass. Any floats or variables passed by value are always preserved as is  (as are the arrays themselves, just not necessarily the values in the array).
See also augmented_primal.
source
EnzymeCore.EnzymeRules.Config — Type
Config{NeedsPrimal, NeedsShadow, Width, Overwritten}
+ConfigWidth{Width} = Config{<:Any,<:Any, Width}
Configuration type to dispatch on in custom reverse rules (see augmented_primal and reverse).
  • NeedsPrimal and NeedsShadow: boolean values specifying whether the primal and shadow (resp.) should be returned. 
  • Width: an integer that specifies the number of adjoints/shadows simultaneously being propagated.
  • Overwritten: a tuple of booleans of whether each argument (including the function itself) is modified between the   forward and reverse pass (true if potentially modified between).
Getters for the four type parameters are provided by needs_primal, needs_shadow, width, and overwritten.
source
EnzymeCore.EnzymeRules.augmented_primal — Function
augmented_primal(::Config, func::Annotation{typeof(f)}, RT::Type{<:Annotation}, args::Annotation...)
Must return an AugmentedReturn type.
  • The primal must be the same type of the original return if needs_primal(config), otherwise nothing.
  • The shadow must be nothing if needs_shadow(config) is false. If width is 1, the shadow should be the same type of the original return. If the width is greater than 1, the shadow should be NTuple{original return, width}.
  • The tape can be any type (including Nothing) and is preserved for the reverse call.
source
EnzymeCore.EnzymeRules.forward — Function
forward(func::Annotation{typeof(f)}, RT::Type{<:Annotation}, args::Annotation...)
Calculate the forward derivative. The first argument func is the callable for which the rule applies to. Either wrapped in a Const), or a Duplicated if it is a closure. The second argument is the return type annotation, and all other arguments are the annotated function arguments.
source
EnzymeCore.EnzymeRules.inactive — Function
inactive(func::typeof(f), args...)
Mark a particular function as always being inactive in both its return result and the function call itself.
source
EnzymeCore.EnzymeRules.inactive_noinl — Function
inactive_noinl(func::typeof(f), args...)
Mark a particular function as always being inactive in both its return result and the function call itself, but do not prevent inlining of the function.
source
EnzymeCore.EnzymeRules.reverse — Function
reverse(::Config, func::Annotation{typeof(f)}, dret::Active, tape, args::Annotation...)
+reverse(::Config, func::Annotation{typeof(f)}, ::Type{<:Annotation), tape, args::Annotation...)
Takes gradient of derivative, activity annotation, and tape. If there is an active return dret is passed as Active{T} with the derivative of the active return val. Otherwise dret is passed as Type{Duplicated{T}}, etc.
source
EnzymeTestUtils.@test_msg — Macro
@test_msg msg condion kws...
This is per Test.@test condion kws... except that if it fails it also prints the msg. If msg=="" then this is just like @test, nothing is printed
Examles
julia> @test_msg "It is required that the total is under 10" sum(1:1000) < 10;
 Test Failed at REPL[1]:1
   Expression: sum(1:1000) < 10
   Problem: It is required that the total is under 10
@@ -159,7 +159,7 @@
   Test Failed at REPL[153]:1
     Expression: sum(1:1000) < 10
      Evaluated: 500500 < 10
-  ERROR: There was an error during testing
source
EnzymeTestUtils.test_forward — Method
test_forward(f, Activity, args...; kwargs...)
Test Enzyme.autodiff of f in Forward-mode against finite differences.
f has all constraints of the same argument passed to Enzyme.autodiff, with several additional constraints:
  • If it mutates one of its arguments, it must return that argument.
Arguments
  • Activity: the activity of the return value of f
  • args: Each entry is either an argument to f, an activity type accepted by autodiff,   or a tuple of the form (arg, Activity), where Activity is the activity type of   arg. If the activity type specified requires a tangent, a random tangent will be   automatically generated.
Keywords
  • fdm=FiniteDifferences.central_fdm(5, 1): The finite differences method to use.
  • fkwargs: Keyword arguments to pass to f.
  • rtol: Relative tolerance for isapprox.
  • atol: Absolute tolerance for isapprox.
  • testset_name: Name to use for a testset in which all tests are evaluated.
Examples
Here we test a rule for a function of scalars. Because we don't provide an activity annotation for y, it is assumed to be Const.
using Enzyme, EnzymeTestUtils
+  ERROR: There was an error during testing
source
EnzymeTestUtils.test_forward — Method
test_forward(f, Activity, args...; kwargs...)
Test Enzyme.autodiff of f in Forward-mode against finite differences.
f has all constraints of the same argument passed to Enzyme.autodiff, with several additional constraints:
  • If it mutates one of its arguments, it must return that argument.
Arguments
  • Activity: the activity of the return value of f
  • args: Each entry is either an argument to f, an activity type accepted by autodiff,   or a tuple of the form (arg, Activity), where Activity is the activity type of   arg. If the activity type specified requires a tangent, a random tangent will be   automatically generated.
Keywords
  • fdm=FiniteDifferences.central_fdm(5, 1): The finite differences method to use.
  • fkwargs: Keyword arguments to pass to f.
  • rtol: Relative tolerance for isapprox.
  • atol: Absolute tolerance for isapprox.
  • testset_name: Name to use for a testset in which all tests are evaluated.
Examples
Here we test a rule for a function of scalars. Because we don't provide an activity annotation for y, it is assumed to be Const.
using Enzyme, EnzymeTestUtils
 
 x, y = randn(2)
 for Tret in (Const, Duplicated, DuplicatedNoNeed), Tx in (Const, Duplicated)
@@ -171,7 +171,7 @@
     Ty in (Const, BatchDuplicated)
 
     test_forward(*, Tret, (x, Tx), (y, Ty))
-end
source
EnzymeTestUtils.test_reverse — Method
test_reverse(f, Activity, args...; kwargs...)
Test Enzyme.autodiff_thunk of f in ReverseSplitWithPrimal-mode against finite differences.
f has all constraints of the same argument passed to Enzyme.autodiff_thunk, with several additional constraints:
  • If it mutates one of its arguments, it must not also return that argument.
  • If the return value is a struct, then all floating point numbers contained in the struct   or its fields must be in arrays.
Arguments
  • Activity: the activity of the return value of f.
  • args: Each entry is either an argument to f, an activity type accepted by autodiff,   or a tuple of the form (arg, Activity), where Activity is the activity type of   arg. If the activity type specified requires a shadow, one will be automatically   generated.
Keywords
  • fdm=FiniteDifferences.central_fdm(5, 1): The finite differences method to use.
  • fkwargs: Keyword arguments to pass to f.
  • rtol: Relative tolerance for isapprox.
  • atol: Absolute tolerance for isapprox.
  • testset_name: Name to use for a testset in which all tests are evaluated.
Examples
Here we test a rule for a function of scalars. Because we don't provide an activity annotation for y, it is assumed to be Const.
using Enzyme, EnzymeTestUtils
+end
source
EnzymeTestUtils.test_reverse — Method
test_reverse(f, Activity, args...; kwargs...)
Test Enzyme.autodiff_thunk of f in ReverseSplitWithPrimal-mode against finite differences.
f has all constraints of the same argument passed to Enzyme.autodiff_thunk, with several additional constraints:
  • If it mutates one of its arguments, it must not also return that argument.
  • If the return value is a struct, then all floating point numbers contained in the struct   or its fields must be in arrays.
Arguments
  • Activity: the activity of the return value of f.
  • args: Each entry is either an argument to f, an activity type accepted by autodiff,   or a tuple of the form (arg, Activity), where Activity is the activity type of   arg. If the activity type specified requires a shadow, one will be automatically   generated.
Keywords
  • fdm=FiniteDifferences.central_fdm(5, 1): The finite differences method to use.
  • fkwargs: Keyword arguments to pass to f.
  • rtol: Relative tolerance for isapprox.
  • atol: Absolute tolerance for isapprox.
  • testset_name: Name to use for a testset in which all tests are evaluated.
Examples
Here we test a rule for a function of scalars. Because we don't provide an activity annotation for y, it is assumed to be Const.
using Enzyme, EnzymeTestUtils
 
 x = randn()
 y = randn()
@@ -180,4 +180,4 @@
 end
Here we test a rule for a function of an array in batch reverse-mode:
x = randn(3)
 for Tret in (Const, Active), Tx in (Const, BatchDuplicated)
     test_reverse(prod, Tret, (x, Tx))
-end
source
Enzyme.API.runtimeActivity! — Method
runtimeActivity!(val::Bool)
Enzyme runs an activity analysis which deduces which values, instructions, etc are necessary to be differentiated and therefore involved in the differentiation procedure. This runs at compile time. However, there may be implementation flaws in this analysis that means that Enzyme cannot deduce that an inactive (const) value is actually const. Alternatively, there may be some data which is conditionally active, depending on which runtime branch is taken. In these cases Enzyme conservatively presumes the value is active.
However, in certain cases, an insufficiently aggressive activity analysis may result in derivative errors – for example by mistakenly using the primal (const) argument and mistaking it for the duplicated shadow. As a result this may result in incorrect results, or accidental updates to the primal.
This flag enables runntime activity which tells all load/stores to check at runtime whether the value they are updating is indeed active (in addition to the compile-time activity analysis). This will remedy these such errors, but at a performance penalty of performing such checks.
It is on the Enzyme roadmap to add a PotentiallyDuplicated style activity, in addition to the current Const and Duplicated styles that will disable the need for this, which does  not require the check when a value is guaranteed active, but still supports runtime-based activity information.
This function takes an argument to set the runtime activity value, true means it is on, and false means off. By default it is off.
source

+end
source
Enzyme.API.runtimeActivity! — Method
runtimeActivity!(val::Bool)

Enzyme runs an activity analysis which deduces which values, instructions, etc are necessary to be differentiated and therefore involved in the differentiation procedure. This runs at compile time. However, there may be implementation flaws in this analysis that means that Enzyme cannot deduce that an inactive (const) value is actually const. Alternatively, there may be some data which is conditionally active, depending on which runtime branch is taken. In these cases Enzyme conservatively presumes the value is active.

However, in certain cases, an insufficiently aggressive activity analysis may result in derivative errors – for example by mistakenly using the primal (const) argument and mistaking it for the duplicated shadow. As a result this may result in incorrect results, or accidental updates to the primal.

This flag enables runntime activity which tells all load/stores to check at runtime whether the value they are updating is indeed active (in addition to the compile-time activity analysis). This will remedy these such errors, but at a performance penalty of performing such checks.

It is on the Enzyme roadmap to add a PotentiallyDuplicated style activity, in addition to the current Const and Duplicated styles that will disable the need for this, which does not require the check when a value is guaranteed active, but still supports runtime-based activity information.

This function takes an argument to set the runtime activity value, true means it is on, and false means off. By default it is off.

source
diff --git a/dev/dev_docs/index.html b/dev/dev_docs/index.html index 6345881358..9e632c58bb 100644 --- a/dev/dev_docs/index.html +++ b/dev/dev_docs/index.html @@ -11,4 +11,4 @@ julia -e "using Pkg; pkg\"add LLVM_full_jll@${LLVM_MAJOR_VER}\"" LLVM_DIR=`julia -e "using LLVM_full_jll; print(LLVM_full_jll.artifact_dir)"` echo "LLVM_DIR=$LLVM_DIR" -cmake ../enzyme/ -G Ninja -DENZYME_EXTERNAL_SHARED_LIB=ON -DLLVM_DIR=${LLVM_DIR} -DLLVM_EXTERNAL_LIT=${LLVM_DIR}/tools/lit/lit.py

Manual build of Julia

cmake ../enzyme/ -G Ninja -DENZYME_EXTERNAL_SHARED_LIB=ON -DLLVM_DIR=${PATH_TO_BUILDDIR_OF_JULIA}/usr/lib/cmake/llvm/
+cmake ../enzyme/ -G Ninja -DENZYME_EXTERNAL_SHARED_LIB=ON -DLLVM_DIR=${LLVM_DIR} -DLLVM_EXTERNAL_LIT=${LLVM_DIR}/tools/lit/lit.py

Manual build of Julia

cmake ../enzyme/ -G Ninja -DENZYME_EXTERNAL_SHARED_LIB=ON -DLLVM_DIR=${PATH_TO_BUILDDIR_OF_JULIA}/usr/lib/cmake/llvm/
diff --git a/dev/generated/autodiff/index.html b/dev/generated/autodiff/index.html index 8572105c59..cc4d7dba8b 100644 --- a/dev/generated/autodiff/index.html +++ b/dev/generated/autodiff/index.html @@ -66,4 +66,4 @@ hess[1][2] == 1.0
true

as well as the second row/column

hess[2][1] == 1.0
 
-hess[2][2] == 0.0
true

This page was generated using Literate.jl.

+hess[2][2] == 0.0
true

This page was generated using Literate.jl.

diff --git a/dev/generated/box/index.html b/dev/generated/box/index.html index fb95e0be34..b01b14bf5b 100644 --- a/dev/generated/box/index.html +++ b/dev/generated/box/index.html @@ -362,4 +362,4 @@ 9.732210923954578e-5 0.0026401658789532625 0.015152571729925521 - 0.03212933056407103

and we get down to a percent difference on the order of $1e^{-5}$, showing Enzyme calculated the correct derivative. Success!

This page was generated using Literate.jl.

+ 0.03212933056407103

and we get down to a percent difference on the order of $1e^{-5}$, showing Enzyme calculated the correct derivative. Success!

This page was generated using Literate.jl.

diff --git a/dev/generated/custom_rule/index.html b/dev/generated/custom_rule/index.html index be55c7233f..62e2739b54 100644 --- a/dev/generated/custom_rule/index.html +++ b/dev/generated/custom_rule/index.html @@ -165,4 +165,4 @@ test_reverse(fun, RT, (x, Tx), (y, Ty)) end end -end
Test.DefaultTestSet("f rules", Any[Test.DefaultTestSet("forward", Any[Test.DefaultTestSet("RT = Const, Tx = Const, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity Const on (::Vector{Float64}, Const), (::Vector{Float64}, Const)", Any[], 7, false, false, true, 1.710845370250021e9, 1.710845372206017e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845367713168e9, 1.710845372206023e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Const, Tx = Const, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity Const on (::Vector{Float64}, Const), (::Vector{Float64}, Duplicated)", Any[], 7, false, false, true, 1.710845372207282e9, 1.710845373498376e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845372206073e9, 1.710845373498381e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Const, Tx = Duplicated, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity Const on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Const)", Any[], 7, false, false, true, 1.710845373499697e9, 1.710845374632341e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845373498423e9, 1.710845374632345e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Const, Tx = Duplicated, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity Const on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 7, false, false, true, 1.710845374633402e9, 1.710845375834725e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845374632383e9, 1.710845375834729e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = DuplicatedNoNeed, Tx = Const, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity DuplicatedNoNeed on (::Vector{Float64}, Const), (::Vector{Float64}, Const)", Any[], 6, false, false, true, 1.710845375835992e9, 1.710845377214238e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845375834772e9, 1.710845377214243e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = DuplicatedNoNeed, Tx = Const, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity DuplicatedNoNeed on (::Vector{Float64}, Const), (::Vector{Float64}, Duplicated)", Any[], 6, false, false, true, 1.710845377215521e9, 1.710845379238219e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845377214282e9, 1.710845379238223e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = DuplicatedNoNeed, Tx = Duplicated, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity DuplicatedNoNeed on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Const)", Any[], 6, false, false, true, 1.710845379239269e9, 1.710845380624968e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845379238252e9, 1.710845380624971e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = DuplicatedNoNeed, Tx = Duplicated, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity DuplicatedNoNeed on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 6, false, false, true, 1.710845380626019e9, 1.710845382364308e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845380625e9, 1.710845382364311e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Duplicated, Tx = Const, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity Duplicated on (::Vector{Float64}, Const), (::Vector{Float64}, Const)", Any[], 7, false, false, true, 1.710845382365358e9, 1.710845383357983e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845382364341e9, 1.710845383357987e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Duplicated, Tx = Const, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity Duplicated on (::Vector{Float64}, Const), (::Vector{Float64}, Duplicated)", Any[], 7, false, false, true, 1.710845383359032e9, 1.710845384438411e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845383358014e9, 1.710845384438415e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Duplicated, Tx = Duplicated, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity Duplicated on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Const)", Any[], 7, false, false, true, 1.710845384439459e9, 1.71084538553415e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710845384438442e9, 1.710845385534154e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Duplicated, Tx = Duplicated, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity Duplicated on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 7, false, false, true, 1.71084538553522e9, 1.710845386705571e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.71084538553418e9, 1.710845386705574e9, false, "custom_rule.md")], 0, false, false, true, 1.710845367713111e9, 1.710845386705576e9, false, "custom_rule.md"), Test.DefaultTestSet("reverse", Any[Test.DefaultTestSet("RT = Active, Tx = Duplicated, Ty = Duplicated, fun = g", Any[Test.DefaultTestSet("test_reverse: g with return activity Active on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 11, false, false, true, 1.710845386846669e9, 1.710845390499541e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_reverse.jl")], 0, false, false, true, 1.710845386705631e9, 1.710845390499548e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Active, Tx = Duplicated, Ty = Duplicated, fun = h", Any[Test.DefaultTestSet("test_reverse: h with return activity Active on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 11, false, false, true, 1.710845390500421e9, 1.710845394291027e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_reverse.jl")], 0, false, false, true, 1.710845390499581e9, 1.710845394291031e9, false, "custom_rule.md")], 0, false, false, true, 1.710845386705596e9, 1.710845394291032e9, false, "custom_rule.md")], 0, false, false, true, 1.710845367709054e9, 1.710845394291033e9, false, "custom_rule.md")

In any package that implements Enzyme rules using EnzymeRules, it is recommended to add EnzymeTestUtils as a test dependency to test the rules.

This page was generated using Literate.jl.

+end
Test.DefaultTestSet("f rules", Any[Test.DefaultTestSet("forward", Any[Test.DefaultTestSet("RT = Const, Tx = Const, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity Const on (::Vector{Float64}, Const), (::Vector{Float64}, Const)", Any[], 7, false, false, true, 1.710855814867708e9, 1.710855816869506e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855812302709e9, 1.710855816869512e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Const, Tx = Const, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity Const on (::Vector{Float64}, Const), (::Vector{Float64}, Duplicated)", Any[], 7, false, false, true, 1.710855816870628e9, 1.710855818199221e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855816869565e9, 1.710855818199228e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Const, Tx = Duplicated, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity Const on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Const)", Any[], 7, false, false, true, 1.710855818200516e9, 1.710855819352931e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855818199272e9, 1.710855819352938e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Const, Tx = Duplicated, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity Const on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 7, false, false, true, 1.710855819354119e9, 1.710855820617671e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855819352984e9, 1.710855820617676e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = DuplicatedNoNeed, Tx = Const, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity DuplicatedNoNeed on (::Vector{Float64}, Const), (::Vector{Float64}, Const)", Any[], 6, false, false, true, 1.710855820618798e9, 1.710855822009339e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855820617723e9, 1.710855822009344e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = DuplicatedNoNeed, Tx = Const, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity DuplicatedNoNeed on (::Vector{Float64}, Const), (::Vector{Float64}, Duplicated)", Any[], 6, false, false, true, 1.710855822010595e9, 1.710855824084041e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855822009389e9, 1.710855824084046e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = DuplicatedNoNeed, Tx = Duplicated, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity DuplicatedNoNeed on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Const)", Any[], 6, false, false, true, 1.710855824085259e9, 1.710855825483249e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855824084077e9, 1.710855825483252e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = DuplicatedNoNeed, Tx = Duplicated, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity DuplicatedNoNeed on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 6, false, false, true, 1.71085582548429e9, 1.710855827240204e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855825483282e9, 1.710855827240208e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Duplicated, Tx = Const, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity Duplicated on (::Vector{Float64}, Const), (::Vector{Float64}, Const)", Any[], 7, false, false, true, 1.710855827241412e9, 1.710855828250792e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855827240238e9, 1.710855828250796e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Duplicated, Tx = Const, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity Duplicated on (::Vector{Float64}, Const), (::Vector{Float64}, Duplicated)", Any[], 7, false, false, true, 1.71085582825204e9, 1.710855829338798e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855828250826e9, 1.710855829338803e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Duplicated, Tx = Duplicated, Ty = Const", Any[Test.DefaultTestSet("test_forward: g with return activity Duplicated on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Const)", Any[], 7, false, false, true, 1.710855829340009e9, 1.71085583045337e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855829338833e9, 1.710855830453374e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Duplicated, Tx = Duplicated, Ty = Duplicated", Any[Test.DefaultTestSet("test_forward: g with return activity Duplicated on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 7, false, false, true, 1.710855830454455e9, 1.710855831631655e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_forward.jl")], 0, false, false, true, 1.710855830453401e9, 1.710855831631658e9, false, "custom_rule.md")], 0, false, false, true, 1.710855812302651e9, 1.710855831631659e9, false, "custom_rule.md"), Test.DefaultTestSet("reverse", Any[Test.DefaultTestSet("RT = Active, Tx = Duplicated, Ty = Duplicated, fun = g", Any[Test.DefaultTestSet("test_reverse: g with return activity Active on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 11, false, false, true, 1.710855831765386e9, 1.710855835315919e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_reverse.jl")], 0, false, false, true, 1.710855831631715e9, 1.710855835315925e9, false, "custom_rule.md"), Test.DefaultTestSet("RT = Active, Tx = Duplicated, Ty = Duplicated, fun = h", Any[Test.DefaultTestSet("test_reverse: h with return activity Active on (::Vector{Float64}, Duplicated), (::Vector{Float64}, Duplicated)", Any[], 11, false, false, true, 1.710855835316635e9, 1.710855839106682e9, false, "/home/runner/work/Enzyme.jl/Enzyme.jl/lib/EnzymeTestUtils/src/test_reverse.jl")], 0, false, false, true, 1.710855835315958e9, 1.710855839106686e9, false, "custom_rule.md")], 0, false, false, true, 1.71085583163168e9, 1.710855839106688e9, false, "custom_rule.md")], 0, false, false, true, 1.710855812298364e9, 1.710855839106688e9, false, "custom_rule.md")

In any package that implements Enzyme rules using EnzymeRules, it is recommended to add EnzymeTestUtils as a test dependency to test the rules.

This page was generated using Literate.jl.

diff --git a/dev/index.html b/dev/index.html index d954740bc3..633bd0efe3 100644 --- a/dev/index.html +++ b/dev/index.html @@ -239,4 +239,4 @@ # output -3.1 + 2.7im

Note: when writing rules for complex scalar functions, in reverse mode one needs to conjugate the differential return, and similarly the true result will be the conjugate of that value (in essence you can think of reverse-mode AD as working in the conjugate space).

+3.1 + 2.7im

Note: when writing rules for complex scalar functions, in reverse mode one needs to conjugate the differential return, and similarly the true result will be the conjugate of that value (in essence you can think of reverse-mode AD as working in the conjugate space).

diff --git a/dev/internal_api/index.html b/dev/internal_api/index.html index e4313ea5f5..c967b879dd 100644 --- a/dev/internal_api/index.html +++ b/dev/internal_api/index.html @@ -1,2 +1,2 @@ -Internal API · Enzyme.jl
Edit on GitHub

Internal API

Note

This is the documentation of Enzymes's internal API. The internal API is not subject to semantic versioning and may change at any time and without deprecation.

+Internal API · Enzyme.jl
Edit on GitHub

Internal API

Note

This is the documentation of Enzymes's internal API. The internal API is not subject to semantic versioning and may change at any time and without deprecation.

diff --git a/dev/pullbacks/index.html b/dev/pullbacks/index.html index 6df29d1a86..e45e174b5a 100644 --- a/dev/pullbacks/index.html +++ b/dev/pullbacks/index.html @@ -27,4 +27,4 @@ Enzyme.autodiff(Reverse, mymul!, Const, Duplicated(R, ∂z_∂R), Duplicated(A, ∂z_∂A), Duplicated(B, ∂z_∂B))
((nothing, nothing, nothing),)

Now we have:

R ≈ A * B            &&
 ∂z_∂A ≈ ∂z_∂R0 * B'   &&  # equivalent to Zygote.pullback(*, A, B)[2](∂z_∂R)[1]
-∂z_∂B ≈ A' * ∂z_∂R0       # equivalent to Zygote.pullback(*, A, B)[2](∂z_∂R)[2]
true

Note that the result of the backpropagation is added to ∂z_∂A and ∂z_∂B, they act as accumulators for gradient information.

+∂z_∂B ≈ A' * ∂z_∂R0 # equivalent to Zygote.pullback(*, A, B)[2](∂z_∂R)[2]
true

Note that the result of the backpropagation is added to ∂z_∂A and ∂z_∂B, they act as accumulators for gradient information.

diff --git a/dev/search/index.html b/dev/search/index.html index 2e92c4ef6d..5347406c0c 100644 --- a/dev/search/index.html +++ b/dev/search/index.html @@ -1,2 +1,2 @@ -Search · Enzyme.jl

Loading search...

    +Search · Enzyme.jl

    Loading search...